Search papers, labs, and topics across Lattice.
This paper introduces MESH, a unified retrieval scaling framework designed to address the Scaling Bias of Heterogeneity in large-scale retrieval systems. By employing a modularized architecture with gated bias correction, MESH effectively partitions the feature space, leading to significant improvements in the retrieval of both fresh and long-tail content. Empirical results demonstrate a 14-fold enhancement in scaling behavior for sparse content, resulting in a 5.5% increase in fresh-item repins on Pinterest and substantial gains in system efficiency and user retention.
MESH achieves a 14x improvement in scaling for fresh content, revolutionizing retrieval efficiency in large-scale recommendation systems.
Optimizing large-scale retrieval hinges on the ability to efficiently surface candidates across diverse content tiers. However, to capture segments such as fresh and long-tail content, modern systems typically resort to a fragmented "zoo" of specialized retrieval models. This operational complexity is attributed to a fundamental challenge in heterogeneous retrieval systems, the Scaling Bias of Heterogeneity, where model capacity gains do not apply equally across diverse content tiers. To bridge this gap, we propose MESH as a unified retrieval scaling framework that mitigates this bias through a modularized architecture integrated with gated bias correction. By partitioning the feature space into independent domains, MESH enforces a structural inductive bias that reduces interference between sparse-item signals and high-frequency engagement features. This protected gradient path leads to improved scaling behavior for sparse content, empirically validated by a 14 times improvement in the power-law scaling exponent for fresh items. In online evaluations on Pinterest's Related Pins platform, a billion scale item-to-item recommendation system, these improvements translate into a +5.5% lift in fresh-item repins, alongside with 55% improvement in funnel efficiency and +0.46% improvement in user retention. Finally, our asynchronous serving strategy ensures production viability by delivering a 2.87 times improvement in system throughput. Our findings suggest MESH as a promising paradigm for consolidating fragmented retrieval infrastructures into more scalable and ecosystem-aware backbones.
