Search papers, labs, and topics across Lattice.
This paper introduces a bi-encoder training framework for sponsored search retrieval that combines semantic relevance and behavioral engagement modeling. They construct a context-rich training target using graded relevance labels from cross-encoders, a multi-channel retrieval prior, and user engagement signals applied only to semantically relevant items. Experiments on Walmart's e-commerce sponsored search demonstrate that this approach outperforms the production system in both offline and online evaluations, yielding gains in relevance and NDCG.
Forget noisy user clicks: training retrievers with semantic relevance as the primary signal, and engagement as a secondary preference signal, significantly boosts search quality.
Modern search systems rely on a fast first stage retriever to fetch relevant items from a massive catalog of items. Deployed search systems often use user engagement signals to supervise bi-encoder retriever training at scale, because these signals are continuously logged from real traffic and require no additional annotation effort. However, engagement is an imperfect proxy for semantic relevance. Items may receive interactions due to popularity, promotion, attractive visuals, titles, or price, despite weak query-item relevance. These limitations are further accentuated in Walmart's e-commerce sponsored search. User engagement on ad items is often structurally sparse because the frequency with which an ad is shown depends on factors beyond relevance such as whether the advertiser is currently running that ad, the outcome of the auction for available ad slots, bid competitiveness, and advertiser budget. Thus, even highly relevant query ad pairs can have limited engagement signals simply due to limited impressions. We propose a bi-encoder training framework for Walmart's sponsored search retrieval in e-commerce that uses semantic relevance as the primary supervision signal, with engagement used only as a preference signal among relevant items. Concretely, we construct a context-rich training target by combining 1. graded relevance labels from a cascade of cross-encoder teacher models, 2. a multichannel retrieval prior score derived from the rank positions and cross-channel agreement of retrieval systems running in production, and 3. user engagement applied only to semantically relevant items to refine preferences. Our approach outperforms the current production system in both offline evaluation and online AB tests, yielding consistent gains in average relevance and NDCG.
