Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores the use of sparse embeddings for cold item recommendation in large-scale recommender systems, addressing the challenges of embedding storage and retrieval for newly added items. By adapting existing cold-start training regimes and introducing a pre-sparsification activation technique, the authors demonstrate that sparse representations not only enhance recommendation accuracy but also reduce storage costs compared to traditional dense embeddings. Comprehensive experiments across four multimodal datasets reveal that these sparse embeddings improve interpretability and maintain robustness in balancing size and accuracy, particularly for users with diverse interests.
Sparse embeddings can significantly outperform dense vectors in cold-start recommendations while slashing storage costs.
The scale and rapid growth of item catalogs in modern digital platforms present significant challenges to recommender system (RS) practitioners. Most RSs use embedding similarity to predict user-item preferences, but embedding storage and low-latency retrieval is challenging in industry-scale catalogs. Furthermore, newly added items do not have corresponding embeddings and cannot be recommended effectively; previous works often tackle this item cold-start problem by generating cold item representations from auxiliary content, such as images or descriptive text, so that user preferences can be predicted without historical interactions. In this paper, we argue that sparse embeddings have notable advantages over standard dense vectors in this content-based cold-start paradigm. We describe how existing cold-start training regimes can be adapted for sparse representation learning, and build on insights from linear attention to design a pre-sparsification activation technique that induces sharpness and denoising effects in learned item-item similarities. We show that the resulting sparse embeddings achieve significant improvements in cold-start recommendation accuracy over dense embeddings at considerably lower storage costs, especially for users with multiple interests. Through comprehensive experiments on four multimodal RS datasets, we also demonstrate the interpretability of sparse content embeddings and their robustness in the trade-off between size and accuracy.