Search papers, labs, and topics across Lattice.
This paper introduces BACH, a novel Bayesian Admixture of Contrastive Heads framework for multi-interest two-tower retrieval that addresses the limitations of traditional hard-routing training by employing variational inference to create a soft mixture of user interests. By allowing each user to dynamically weight their interests during retrieval, BACH mitigates routing collapse and enhances the representation of diverse user preferences. Experimental results on large-scale benchmarks, including MovieLens-20M, Taobao, and Netflix, demonstrate that BACH consistently outperforms both hard-routing multi-interest models and single-vector baselines across various head counts, particularly by scoring candidates based on their best head.
BACH not only prevents routing collapse but also reveals the nuanced importance of user interests, leading to superior retrieval performance across diverse datasets.
Two-tower retrievers compress each user into a single embedding, limiting their ability to serve diverse interests. Multi-interest models give each user several heads scored by a maximum inner product, but their hard-routing training under-utilizes heads (routing collapse) and gives no per-user estimate of how much each interest matters for serving. We present \textbf{BACH} (\emph{Bayesian Admixture of Contrastive Heads}), which casts multi-interest two-tower retrieval as a per-user mixture over the heads, fit by variational inference. The soft mixture trains every head (mitigating collapse), produces a per-user weighting of the interests that is reused at serving, and admits a shared global-codebook variant with precomputable retrieval. On three large-scale benchmarks, MovieLens-20M, Taobao, and Netflix, BACH improves top-of-ranking retrieval over hard-routing multi-interest and single-vector baselines at every head count; we further find that scoring every candidate by its best head, consistent with serving, outperforms the usual target-routed training, and that BACH improves further still.
