Search papers, labs, and topics across Lattice.
This paper revitalizes the classic two-tower retrieval architecture by employing large language models (LLMs) as semantic representation backbones instead of generative engines, addressing computational overhead and grounding challenges in web-scale applications. The authors introduce several innovations, including a shared LLM encoder, EOS token pooling, and cross-dataset transfer learning, which collectively enhance retrieval efficiency and effectiveness. Extensive evaluations reveal that their LLM-native two-tower framework achieves state-of-the-art performance while maintaining high resilience to model staleness and scalability in real-world systems.
Traditional two-tower architectures can outperform state-of-the-art models when enhanced with LLM-driven semantic representations.
Large Language Models (LLMs) have emerged as powerful assets for recommender systems. However, deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges. In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines. We introduce an LLM-native two-tower framework engineered for high-throughput, large-scale retrieval. Our architecture introduces several key innovations: a shared LLM encoder for joint user-item modeling, End-Of-Sentence (EOS) token pooling for compact sequence embedding, cross-dataset transfer learning, knowledge distillation from powerful cross-encoder teachers, and latent reasoning within the user tower. Extensive evaluation across three public benchmarks demonstrates that cross-encoder architecture outperforms current state-of-the-art (SoTA) models, while the efficient two-tower student achieves SoTA-comparable retrieval performance. Furthermore, experiments on internal large-scale production systems yield substantial topline retrieval improvements along with high resilience to model staleness and superior data scaling. Our findings demonstrate that when augmented with modern representation learning, the traditional two-tower paradigm remains an exceptionally competitive and practical solution for industrial retrieval systems.
Amazon Science