Search papers, labs, and topics across Lattice.
This paper introduces GEM, a generative embedding model that enhances information retrieval by integrating reasoning about user intent and relevance criteria directly into the retrieval process. By appending an embedding token that encodes enriched context after reasoning over the query, GEM effectively bridges the gap between user expression and retriever interpretation. Evaluated on reasoning-intensive tasks, GEM outperforms traditional retrieval methods and its non-reasoning variant, demonstrating significant improvements in performance with the potential for scalable compute at test time.
GEM's innovative approach to integrating reasoning into retrieval processes significantly boosts performance, outperforming conventional methods and its own non-reasoning variant.
Modern LLMs excel at reasoning and instruction following, enabling users to express complex and diverse information needs. However, conventional retrievers largely rely on surface-level matching between queries and documents, resulting in a growing gap between how users express their needs and how retrievers interpret them. In this paper, we present GEM, a generative embedding model that augments retrieval through its own knowledge by explicitly reasoning about user intent and relevance criteria. GEM unifies generation and embedding within a single model: it first reasons over the query, then appends an embedding token to encode the enriched context for retrieval. \zhili{Evaluated on reasoning-intensive and instruction-following retrieval tasks, GEM demonstrates the effectiveness of its reasoning-augmented retrieval, outperforming its non-reasoning variant and matching baselines using substantially larger models.} Furthermore, GEM's generative nature allows test-time compute scaling via prompting to further enhance retrieval performance. Our code is available at: https://anonymous.4open.science/r/GEM.
