Search papers, labs, and topics across Lattice.
The paper introduces Query-as-Anchor, a framework that shifts user modeling from static encoding to dynamic, query-aware synthesis using Large Language Models (LLMs) to address the challenge of balancing robust universality with acute task-sensitivity in industrial-scale user representation learning. They construct UserU, a large-scale pre-training dataset aligning multi-modal behavioral sequences with user understanding semantics, and propose a Q-Anchor Embedding architecture integrating hierarchical encoders into dual-tower LLMs via joint contrastive-autoregressive optimization. Evaluations on Alipay benchmarks and online A/B testing demonstrate state-of-the-art performance, scalability, and efficient deployment.
LLMs can be adapted for scenario-specific user representation by conditioning on queries, achieving state-of-the-art performance on Alipay benchmarks and demonstrating practical effectiveness in online A/B testing.
Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.
