Search papers, labs, and topics across Lattice.
This paper introduces TransRetrieval, a Transformer-based retrieval framework designed to address the challenges of feature heterogeneity in recommendation systems by applying scaling laws effectively. The method employs weighted average aggregation to restore the homogeneous-token assumption, alongside target token compression that reduces per-candidate FLOPs by 85% while maintaining cross-attention capabilities. Experimental results on a large industrial dataset and the KuaiRand benchmark demonstrate significant improvements in recall and a 2.53% increase in platform revenue during online A/B testing, showcasing the framework's practical impact in real-world applications.
TransRetrieval achieves a 2.53% revenue boost while slashing computational costs by 85%, revolutionizing recommendation retrieval in industrial settings.
Applying scaling laws to recommendation retrieval is hindered by feature heterogeneity: naively stacking Transformer layers yields diminishing returns because heterogeneous fields produce severe token-norm divergence. We present TransRetrieval, a Transformer-based retrieval framework that scales with both computational budget and cross-domain data. The key enabler is (1) weighted average aggregation, which restores the homogeneous-token assumption Transformers rely on. Building on this, we introduce (2) target token compression that cuts per-candidate FLOPs by 85% while preserving cross-attention expressiveness, and (3) position-style domain embeddings that unify multiple domains at negligible additional cost, turning cross-domain data into a scaling asset. On a 40-billion-interaction industrial dataset and the public KuaiRand benchmark, scaling compute from 0.1 to 2 MFLOPs per target yields +19.3/+22.2 pt Recall@2000, confirming robust log-linear scaling. In online A/B tests, TransRetrieval lifts platform revenue by 2.53% under the same end-to-end latency constraint as the production baseline.
