R2LED: Equipping Retrieval and Refinement in Lifelong User Modeling with Semantic IDs for CTR Prediction

2026-02-06

AI Summary

This work addresses the limitations of existing lifelong user modeling approaches, which are prone to noise from skewed data distributions during retrieval and lack deep semantic understanding in refinement. To overcome these challenges, we propose R2LED, a novel framework that integrates lightweight multi-granularity semantic IDs (SIDs) throughout the entire modeling pipeline. R2LED enhances interest coverage and suppresses noise via multi-routed hybrid retrieval, and effectively bridges collaborative signals with semantic representations through target-aware cross-attention and gating mechanisms in a dual-level fusion refinement module. Experimental results on two public datasets demonstrate that R2LED significantly outperforms state-of-the-art methods, achieving consistent improvements in both click-through rate prediction accuracy and inference efficiency.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Lifelong user modeling, which leverages users'long-term behavior sequences for CTR prediction, has been widely applied in personalized services. Existing methods generally adopted a two-stage"retrieval-refinement"strategy to balance effectiveness and efficiency. However, they still suffer from (i) noisy retrieval due to skewed data distribution and (ii) lack of semantic understanding in refinement. While semantic enhancement, e.g., LLMs modeling or semantic embeddings, offers potential solutions to these two challenges, these approaches face impractical inference costs or insufficient representation granularity. Obsorbing multi-granularity and lightness merits of semantic identity (SID), we propose a novel paradigm that equips retrieval and refinement in Lifelong User Modeling with SEmantic IDs (R2LED) to address these issues. First, we introduce a Multi-route Mixed Retrieval for the retrieval stage. On the one hand, it captures users'interests from various granularities by several parallel recall routes. On the other hand, a mixed retrieval mechanism is proposed to efficiently retrieve candidates from both collaborative and semantic views, reducing noise. Then, for refinement, we design a Bi-level Fusion Refinement, including a target-aware cross-attention for route-level fusion and a gate mechanism for SID-level fusion. It can bridge the gap between semantic and collaborative spaces, exerting the merits of SID. The comprehensive experimental results on two public datasets demonstrate the superiority of our method in both performance and efficiency. To facilitate the reproduction, we have released the code online https://github.com/abananbao/R2LED.

Problem

• lifelong user modeling
• CTR prediction
• retrieval-refinement
• semantic understanding
• noisy retrieval

Innovation

• Semantic ID
• Lifelong User Modeling
• Retrieval-Refinement
• Multi-granularity Retrieval
• Bi-level Fusion