KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao

2026-03-24

AI Summary

This work addresses the semantic collapse in large language models for personalized search, where conflicts between pre-trained knowledge and user behavioral objectives degrade performance. To mitigate this, the authors propose the KARMA framework, which optimizes user interest representations while introducing, for the first time, a semantic decodability regularization. This regularization jointly enforces two objectives—historical-conditioned generation and embedding-conditioned reconstruction—to alleviate attention “sinking” and align pre-trained semantic knowledge with user behavior goals. Evaluated on Taobao’s search system, KARMA achieves significant gains: HR@200 improves by up to 22.5, with HR increases of 1.86 and 2.51 in pre-ranking and recall stages, respectively; ranking-stage CTR AUC rises by 0.25; and online item click-through rate improves by 0.5%.

Technology Category

• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Applications of semantic technologies for improving search, browsing, recommendation, personalization

Abstract

Large Language Models (LLMs) are equipped with profound semantic knowledge, making them a natural choice for injecting semantic generalization into personalized search systems. However, in practice we find that directly fine-tuning LLMs on industrial personalized tasks (e.g. next item prediction) often yields suboptimal results. We attribute this bottleneck to a critical Knowledge--Action Gap: the inherent conflict between preserving pre-trained semantic knowledge and aligning with specific personalized actions by discriminative objectives. Empirically, action-only training objectives induce Semantic Collapse, such as attention ``sinks''. This degradation severely cripples the LLM's generalization, failing to bring improvements to personalized search systems. We propose KARMA (Knowledge--Action Regularized Multimodal Alignment), a unified framework that treats semantic reconstruction as a train-only regularizer. KARMA optimizes a next-interest embedding for retrieval (Action) while enforcing semantic decodability (Knowledge) through two complementary objectives: (i) history-conditioned semantic generation, which anchors optimization to the LLM's native next-token distribution, and (ii) embedding-conditioned semantic reconstruction, which constrains the interest embedding to remain semantically recoverable. On Taobao search system, KARMA mitigates semantic collapse (attention-sink analysis) and improves both action metrics and semantic fidelity. In ablations, semantic decodability yields up to +22.5 HR@200. With KARMA, we achieve +0.25 CTR AUC in ranking, +1.86 HR in pre-ranking and +2.51 HR in recalling. Deployed online with low inference overhead at ranking stage, KARMA drives +0.5% increase in Item Click.

Problem

• Knowledge-Action Gap
• Semantic Collapse
• Personalized Search
• Large Language Models
• Industrial Recommendation

Innovation

• Knowledge-Action Gap
• Semantic Collapse
• Semantic Decodability
• Multimodal Alignment
• Personalized Search