Search papers, labs, and topics across Lattice.
This paper introduces SequenceO1, an end-to-end framework designed for ultra-long user behavior sequence modeling in recommendation systems, capable of handling sequences of up to 100K interactions. By employing a compress-then-reason design, it utilizes Sketch Attention for efficient representation of user histories and Stacked Target-to-History Cross Attention to balance short-term and long-term preferences. The framework's innovative caching and batching techniques lead to significant improvements in both offline and online performance, demonstrating a scalable approach to managing extensive sequence data in real-time applications.
SequenceO1 achieves efficient ultra-long sequence modeling by compressing user histories while maintaining high performance, revolutionizing recommendation systems at scale.
Modeling long-term user behavior is central to sequential recommendation and billion-scale industrial recommender systems, yet production ranking models operate under strict latency, memory, communication, and training-throughput constraints. At the 100K scale, the challenge extends beyond attention complexity: raw sequence features must be stored, transferred, and repeatedly processed during training and online serving. Existing approaches based on history truncation, multi-stage behavior retrieval, compressed lifelong histories, or train-short/infer-long extrapolation either weaken end-to-end optimization or retain substantial length-dependent cost. We present SequenceO1, an end-to-end framework for ultra-long user behavior sequence modeling, deployed at full traffic on Douyin with histories of up to 100K interactions. SequenceO1 follows a compress-then-reason design. Its Sketch Attention (SA) uses learnable prototypes and prototype-wise normalization to compress the raw history into a fixed-size, target-agnostic user representation. Target-conditioned Stacked Target-to-History Cross Attention (STCA) then models complementary time scales: a recent 10K suffix for short-term interests and the compact sketch for long-term preferences. To make training and inference practical, SequenceO1 combines low-rank user representation caching, multi-request user-level batching, pipeline lift, and a fused FlashSA kernel to amortize feature storage, communication, and computation across targets, training instances, and consecutive requests. Production experiments show consistent offline and online gains, while the compact cached sketch retains most of the benefit of directly scaling end-to-end sequence ranking to 100K. These results provide a practical model-system approach to efficient attention, sequence compression, and scalable long-sequence and long-context recommendation systems.
