Search papers, labs, and topics across Lattice.
This paper tackles the challenge of efficiently deploying large-scale retrieval models for recommendation by learning a hierarchical index structure using cross-attention and residual quantization. The learned index enables faster search with minimal loss of accuracy by organizing the model's memory hierarchically. Deployed at Meta for advertisement recommendations, the approach also reveals that fine-tuning on data represented by intermediate index nodes further boosts performance, effectively implementing "test-time training."
Meta's new hierarchical indexing method lets you deploy massive recommendation models without sacrificing speed or accuracy, and it turns out the index itself highlights a high-quality subset of data perfect for test-time training.
The increase in data volume, computational resources, and model parameters during training has led to the development of numerous large-scale industrial retrieval models for recommendation tasks. However, effectively and efficiently deploying these large-scale foundational retrieval models remains a critical challenge that has not been fully addressed. Common quick-win solutions for deploying these massive models include relying on offline computations (such as cached user dictionaries) or distilling large models into smaller ones. Yet, both approaches fall short of fully leveraging the representational and inference capabilities of foundational models. In this paper, we explore whether it is possible to learn a hierarchical organization over the memory of foundational retrieval models. Such a hierarchical structure would enable more efficient search by reducing retrieval costs while preserving exactness. To achieve this, we propose jointly learning a hierarchical index using cross-attention and residual quantization for large-scale retrieval models. We also present its real-world deployment at Meta, supporting daily advertisement recommendations for billions of Facebook and Instagram users. Interestingly, we discovered that the intermediate nodes in the learned index correspond to a small set of high-quality data. Fine-tuning the model on this set further improves inference performance, and concretize the concept of"test-time training"within the recommendation system domain. We demonstrate these findings using both internal and public datasets with strong baseline comparisons and hope they contribute to the community's efforts in developing the next generation of foundational retrieval models.
