Search papers, labs, and topics across Lattice.
LoopFM addresses the knowledge transfer bottleneck from large foundation models (FMs) to smaller vertical models (VMs) in recommendation systems by using historical FM embeddings as input features for VMs. This approach avoids the limitations of single-scalar knowledge distillation (KD) and the need for real-time FM inference. Experiments on public benchmarks and industrial-scale systems demonstrate significant AUC and conversion improvements, showing LoopFM's ability to substantially increase the knowledge transfer ratio compared to KD alone.
Doubling the knowledge transfer ratio from trillion-parameter foundation models to recommendation systems is now possible without real-time FM inference, unlocking significant conversion improvements in large-scale industrial settings.
Knowledge distillation (KD) transfers a single scalar prediction from a large foundation model (FM) to compact vertical models (VMs), suffering from diminishing transfer ratio -- the fraction of FM improvement captured by the VM -- as a single scalar cannot convey the rich intermediate knowledge that larger FMs learn. To address this bottleneck, we propose LoopFM (Learning frOm HistOrical ReP*resentations of FM), a framework that opens a high-bandwidth transfer channel by structuring FM intermediate embeddings as input features (e.g., user history sequence) for downstream VMs, without requiring real-time FM inference at serving and architectural coupling between FM and VM. We provide a theoretical framework for LoopFM with a gain decomposition and transfer-ratio analysis. On three public benchmarks, LoopFM demonstrates strong AUC improvements (e.g., 6\%+ on TaobaoAd) and complementary knowledge transfer capability with KD. On industrial-scale systems (billions of examples, trillion-parameter FMs), LoopFM approximately doubles the knowledge transfer ratio on top of KD, delivering a +0.5\% conversion improvement in Y1H1, and a +1.03\% and +1.22\% conversion improvement from two individual launches respectively in Y1H2.
