Search papers, labs, and topics across Lattice.
This paper introduces CCFormer, an innovative Transformer architecture designed to enhance industrial recommendation systems by efficiently managing cross-field feature interactions and long-sequence modeling. By employing a hierarchical sequence compression strategy and feature-field separated cross attention, CCFormer significantly reduces computational overhead while maintaining high-quality recommendations. Experimental results show that CCFormer outperforms existing models, achieving notable improvements in click-through rates and advertising revenue in real-world applications at Tencent.
CCFormer delivers a 3.57% increase in click-through rates and a 1.71% boost in advertising revenue, all while cutting model training time by over 2x.
Recent studies in industrial recommendation systems have demonstrated that sequential recommendation models built upon self-attention can benefit from predictable scaling laws by increasing sequence length and model capacity. However, practical recommender systems impose strict latency and resource constraints, making it challenging to balance computational overhead with fine-grained feature interaction. In this paper, we propose CCFormer, an efficient Transformer backbone that unifies cross-field feature interaction and compressed long-sequence modeling for industrial recommendation. Specifically, CCFormer combines feature-field separated cross attention with long-sequence subspace token mixing to exploit long-term preference signals across heterogeneous feature domains. A hierarchical sequence compression strategy with progressively expanded receptive fields enables efficient long-sequence modeling with reduced information loss. Extensive experiments on two public benchmarks and a large-scale industrial dataset demonstrate that CCFormer consistently outperforms state-of-the-art baselines. Online A/B tests in a video recommendation scenario and an advertising ranking scenario at Tencent further validate its industrial practicality, yielding a 3.57% CTR gain and a 1.71% advertising revenue lift, respectively, while accelerating model training by 2.21x over the strong HSTU baseline. CCFormer has been fully deployed in Tencent's production recommendation system, serving the main traffic of both scenarios.
