Search papers, labs, and topics across Lattice.
The paper introduces a residual quantization (RQ)-based sequence learning framework for continuous value prediction in recommendation systems, addressing the limitations of existing generative approaches that rely on rigid parametric distribution assumptions. The framework represents continuous values as a sum of ordered quantization codes predicted recursively from coarse to fine granularity. By introducing a representation learning objective that aligns RQ code embedding space with the ordinal structure of target values, the model captures continuous representations for quantization codes, leading to improved prediction accuracy.
Ditch rigid distribution assumptions: a novel residual quantization approach predicts continuous values by recursively refining quantization codes, outperforming SOTA in recommendation tasks.
Continuous value prediction plays a crucial role in industrial-scale recommendation systems, including tasks such as predicting users'watch-time and estimating the gross merchandise value (GMV) in e-commerce transactions. However, it remains challenging due to the highly complex and long-tailed nature of the data distributions. Existing generative approaches rely on rigid parametric distribution assumptions, which fundamentally limits their performance when such assumptions misalign with real-world data. Overly simplified forms cannot adequately model real-world complexities, while more intricate assumptions often suffer from poor scalability and generalization. To address these challenges, we propose a residual quantization (RQ)-based sequence learning framework that represents target continuous values as a sum of ordered quantization codes, predicted recursively from coarse to fine granularity with diminishing quantization errors. We introduce a representation learning objective that aligns RQ code embedding space with the ordinal structure of target values, allowing the model to capture continuous representations for quantization codes and further improving prediction accuracy. We perform extensive evaluations on public benchmarks for lifetime value (LTV) and watch-time prediction, alongside a large-scale online experiment for GMV prediction on an industrial short-video recommendation platform. The results consistently show that our approach outperforms state-of-the-art methods, while demonstrating strong generalization across diverse continuous value prediction tasks in recommendation systems.
