ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling

2025-08-19

AI Summary

To address the challenge of jointly leveraging full-sequence information and extracting highly relevant long-term user interests in long-term behavior modeling, this paper proposes ENCODE: an offline-online collaborative two-stage framework. In the offline stage, metric learning-driven dimensionality reduction and behavioral sequence clustering are employed to precisely distill compact, semantically consistent representations of users’ long-term interests. In the online stage, a unified relevance scoring mechanism enables rapid matching between user interest embeddings and candidate items, ensuring millisecond-level latency. Key innovations include: (1) integrating metric learning into dimensionality reduction to enhance both clustering efficiency and semantic coherence; and (2) designing an end-to-end unified relevance matching module to improve prediction stability. Extensive experiments on multiple public benchmarks demonstrate that ENCODE significantly outperforms existing state-of-the-art methods in click-through rate prediction while maintaining sub-10ms online inference latency.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Long-term user behavior sequences are a goldmine for businesses to explore users' interests to improve Click-Through Rate. However, it is very challenging to accurately capture users' long-term interests from their long-term behavior sequences and give quick responses from the online serving systems. To meet such requirements, existing methods "inadvertently" destroy two basic requirements in long-term sequence modeling: R1) make full use of the entire sequence to keep the information as much as possible; R2) extract information from the most relevant behaviors to keep high relevance between learned interests and current target items. The performance of online serving systems is significantly affected by incomplete and inaccurate user interest information obtained by existing methods. To this end, we propose an efficient two-stage long-term sequence modeling approach, named as EfficieNt Clustering based twO-stage interest moDEling (ENCODE), consisting of offline extraction stage and online inference stage. It not only meets the aforementioned two basic requirements but also achieves a desirable balance between online service efficiency and precision. Specifically, in the offline extraction stage, ENCODE clusters the entire behavior sequence and extracts accurate interests. To reduce the overhead of the clustering process, we design a metric learning-based dimension reduction algorithm that preserves the relative pairwise distances of behaviors in the new feature space. While in the online inference stage, ENCODE takes the off-the-shelf user interests to predict the associations with target items. Besides, to further ensure the relevance between user interests and target items, we adopt the same relevance metric throughout the whole pipeline of ENCODE. The extensive experiment and comparison with SOTA have demonstrated the effectiveness and efficiency of our proposed ENCODE.

Problem

• Model long-term user behavior sequences for accurate interest capture
• Balance online system efficiency with high-performance modeling
• Address trade-offs between sequence completeness and relevance extraction

Innovation

• Two-stage modeling with offline clustering and online inference
• Metric learning-based dimension reduction for efficient clustering
• Unified relevance metric throughout the entire modeling pipeline