Search papers, labs, and topics across Lattice.
This paper introduces DrIG, a novel framework for generative information retrieval that utilizes Dual-role Identifiers to enhance multimodal retrieval across text, image, and mixed content. By employing a dual approach where identifiers serve both sequential and set-based roles, DrIG mitigates issues related to prefix-level errors and local optima while improving retrieval accuracy compared to traditional dense-vector methods. Experimental results demonstrate that DrIG outperforms existing generative multimodal baselines and achieves a favorable efficiency-effectiveness balance against leading dense retrieval systems on the M-BEIR benchmark and text-to-image datasets.
Dual-role Identifiers in DrIG not only enhance retrieval accuracy but also tackle local optima challenges, revolutionizing multimodal information retrieval.
Generative information retrieval (GIR) has emerged as a compelling alternative to the conventional index-retrieve-then-rank retrieval pipeline by training a generator to produce the identifiers of relevant items directly. Despite its promise, a number of open challenges still remain. First, constrained left-to-right decoding is vulnerable to prefix-level errors and local optima. Second, most prior GIR research remains largely unimodal, leaving instruction-aware retrieval across text, image, and mixed image-text items underexplored. Third, although discrete identifier-based GIR offers higher efficiency, its retrieval accuracy still lags behind that of the cutting-edge dense-vector-based retrieval methods. Motivated by these challenges, we propose DrIG, a novel Generative framework for universal multimodal retrieval featuring Dual-role Identifiers, which supports diverse retrieval tasks across multiple modalities and domains. Each candidate is assigned a single residual-quantized identifier that serves two complementary roles. In its sequential role, the identifier is decoded autoregressively, where the first token explicitly models modality and the remaining tokens capture progressively finer semantics. In its set-based role, the same tokens are reinterpreted as an unordered set to provide a prefix-independent relevance prior, which guides constrained beam search and alleviates local-optimum errors. Extensive experiments on the M-BEIR benchmark and the text-to-image evaluation datasets show that:(1)DrIG consistently outperforms state-of-the-art generative multimodal baselines across diverse tasks, while hybrid reranking achieves a favorable efficiency-effectiveness trade-off against strong dense retrievers. (2)Ablation and scaling analyses reveal how the base LMM, beam size, reranking depth, and fusion strategy affect retrieval performance, providing practical guidance for system design.
