Search papers, labs, and topics across Lattice.
This paper introduces Sparse Activation-based Residual Soft Quantization (SA-RSQ), a novel framework designed to optimize the storage and latency challenges associated with high-dimensional multimodal features in recommender systems. By employing Top-K sparse routing and softmax weights, SA-RSQ effectively decouples per-item storage from codebook dimensionality, allowing for efficient gradient propagation without the limitations of traditional quantization methods. Experimental results on a food-delivery advertising dataset reveal significant improvements in reconstruction performance and click-through rates (CTR), with an observed lift of +2.51% in CTR and +3.66% in cost per mille (CPM) during online testing.
SA-RSQ achieves a remarkable balance between compact storage and high-quality representation, leading to significant performance boosts in real-world recommender systems.
Deploying high-dimensional multimodal features in industrial recommender systems incurs substantial storage and latency overhead. Hard quantization is compact but introduces boundary distortion, whereas dense soft quantization couples representation quality to the limited storage budget. We propose Sparse Activation-based Residual Soft Quantization (SA-RSQ), which uses Top-K sparse routing and softmax weights to store compact (Index, Probability) tuples. The stored tuples decouple per-item storage from codebook dimensionality; for a fixed selected support, gradients propagate through the routing weights and weighted reconstruction without relying on a straight-through estimator. Experiments on a proprietary food-delivery advertising dataset show favorable reconstruction-performance and CTR trade-offs across storage budgets of 8-48 bytes per item. A preliminary Next-Distribution Prediction study and a one-week online A/B test further demonstrate the practical potential of SA-RSQ, with relative lifts of +2.51% in CTR and +3.66% in CPM.
