Search papers, labs, and topics across Lattice.
This paper introduces EMMETT, a novel algorithmic framework that synthesizes classifiers on-the-fly for zero-shot retrieval tasks, leveraging existing classifiers for observed items to enhance representation capacity. The method addresses the limitations of conventional Siamese-style approaches, which struggle with the rapid influx of novel items due to their small encoder capacity. Experimental results show that IRENE, an implementation of EMMETT, significantly boosts zero-shot retrieval accuracy by up to 15 percentage points in Recall@10 and increases ad click-through rates by 4.2% in real-world applications.
IRENE not only enhances zero-shot retrieval accuracy but also drives a 4.2% increase in ad click-through rates in live environments.
We develop accurate and efficient solutions for large-scale retrieval tasks where novel (zero-shot) items can arrive continuously at a rapid pace. Conventional Siamese-style approaches embed both queries and items through a small encoder and retrieve the items lying closest to the query. While this approach allows efficient addition and retrieval of novel items, the small encoder lacks sufficient capacity for the necessary world knowledge in complex retrieval tasks. The extreme classification approaches have addressed this by learning a separate classifier for each item observed in the training set which significantly increases the representation capacity of the model. Such classifiers outperform Siamese approaches on observed items, but cannot be trained for novel items due to data and latency constraints. To bridge these gaps, this paper develops: (1) A new algorithmic framework, EMMETT, which efficiently synthesizes classifiers on-the-fly for novel items, by relying on the readily available classifiers for observed items; (2) A new algorithm, IRENE, which is a simple and effective instance of EMMETT that is specifically suited for large-scale deployments, and (3) A new theoretical framework for analyzing the generalization performance in large-scale zero-shot retrieval which guides our algorithm and training related design decisions. Comprehensive experiments are conducted on a wide range of retrieval tasks which demonstrate that IRENE improves the zero-shot retrieval accuracy by up to 15% points in Recall@10 when added on top of leading encoders. Additionally, on an online A/B test in a large-scale ad retrieval task in a major search engine, IRENE improved the ad click-through rate by 4.2%. Lastly, we validate our design choices through extensive ablative experiments. The source code for IRENE is available at https://aka.ms/irene.
Microsoft Research