Search papers, labs, and topics across Lattice.
This paper introduces ReST, a recommendation-native Transformer scaling framework designed to address the unique challenges of behavior-sequence modeling in industrial ranking systems. By implementing a sequence encoder with dual-gated attention and a factorized ranking approach, ReST significantly enhances accuracy and scalability compared to traditional LLM-style Transformers. In a real-world application, ReST improved online AUC by 1.31% and increased a core revenue metric by 11.93% within strict latency constraints, demonstrating its effectiveness in production environments.
ReST shows that tailored Transformer architectures can outperform conventional models in industrial recommendation tasks, achieving significant gains in both accuracy and revenue.
Scaling Transformers has driven large gains in language modeling, but transplanting this to behavior-sequence modeling in production ranking is challenging: recommendation differs in signal quality, where behavior sequences are noisy, temporally irregular, and sparsely supervised, and in computation asymmetry, where each request scores many candidates against one shared user history under tight latency budgets. We propose ReST, a recommendation-native Transformer scaling framework. For signal quality, it introduces a sequence encoder with dual-gated attention, rotary positional and temporal embedding, stabilized residual normalization, and training-only auxiliary objectives. For computation asymmetry, it factorizes ranking into a heavy reusable encoder and a lightweight cross decoder with projection-free KV attention and token-specific parameterization, coupling user-level shared-prefix training with shared-prefix serving for compute-once, decode-many-times ranking. Across industrial and public benchmarks, ReST achieves higher accuracy and scales more consistently along sequence length, depth, and width, where LLM-style Transformer blocks saturate. A one-week online A/B test on a production advertising platform improves online AUC by 1.31% and lifts a core revenue metric by 11.93% within a 50 ms P99 budget; ReST has since been fully deployed in production, showing that behavior-sequence scaling remains a promising, under-exploited axis for production ranking.
