Search papers, labs, and topics across Lattice.
This paper addresses the limitations of dataset-specific fine-tuning in Referring Expression Comprehension (REC) by proposing a unified open-vocabulary grounding approach that mitigates representation degeneration. The authors introduce the Modulated Attention-Contrastive Head (mACH) and a text-conditioned JEPA auxiliary stream to enhance token-level vision-language alignment while maintaining representation diversity. Their framework, validated through extensive experiments, achieves competitive performance across various REC benchmarks and demonstrates strong generalization capabilities without requiring adaptation to specific datasets.
A single generalist model can achieve state-of-the-art performance in Referring Expression Comprehension while generalizing across diverse datasets without fine-tuning.
Referring Expression Comprehension (REC) is commonly studied under dataset-specific fine-tuning, resulting in specialist models with limited cross-dataset generalization. In this work, we revisit REC from the perspective of unified open-vocabulary grounding and identify representation degeneration as a key obstacle to scaling a single generalist model. To preserve representation diversity, we propose a holistic data-model co-design framework. Architecturally, we introduce the Modulated Attention-Contrastive Head (mACH) for efficient token-level vision-language alignment and a text-conditioned JEPA auxiliary stream that provides complementary gradient support to preserve alignment-active representations without inference overhead. On the data side, we introduce Objects365-Caption, enriching Objects365 with context-aware referring expressions for large-scale language supervision. We further provide a theoretical analysis showing that complementary gradient subspaces preserve alignment capacity and thereby scale representation diversity. Extensive experiments demonstrate that our single-checkpoint framework achieves highly competitive performance on standard REC benchmarks while exhibiting strong generalization across heterogeneous grounding datasets without benchmark-specific adaptation.
