Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

2026-02-16

AI Summary

Existing user representation methods produce static, task-agnostic embeddings that struggle to balance the conflicting demands of cross-scenario generalization and task-specific sensitivity, while also being vulnerable to noise and modality conflicts in multi-source heterogeneous data. To address these limitations, this work proposes the Query-as-Anchor framework, which reformulates user modeling as a dynamic, query-aware synthesis paradigm. Leveraging a self-constructed industrial-scale pretraining dataset, UserU, the approach employs a dual-tower large language model architecture with a hierarchical coarse-to-fine encoder. It further integrates a cluster-based soft prompt tuning mechanism to align modality representations with scenario-specific contexts and utilizes KV caching to accelerate inference. The method achieves state-of-the-art performance across ten Alipay industrial benchmarks, and large-scale online A/B tests confirm its effectiveness and deployment efficiency.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.

Problem

• user representation learning
• task-sensitive modeling
• multi-source heterogeneity
• modality conflict
• industrial-scale embedding

Innovation

• Query-as-Anchor
• User Representation
• Large Language Model
• Soft Prompt Tuning
• Multi-modal Behavioral Sequences