Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

2026-01-14

AI Summary

This work addresses the challenges of gradient conflict and general knowledge drift in large language models (LLMs) when unifying search and recommendation tasks. To this end, the authors propose GEMS (Gradient-Efficient Multi-Subspace tuning), a parameter-efficient fine-tuning framework grounded in a conditional generation paradigm that jointly models both tasks within a single LLM. GEMS mitigates inter-task gradient interference through multi-subspace decomposition and preserves pre-trained general knowledge by incorporating null-space projection constraints that prevent catastrophic forgetting during fine-tuning. Extensive experiments demonstrate that GEMS significantly outperforms state-of-the-art methods across multiple benchmark datasets, achieving superior performance in both search and recommendation tasks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.

Problem

• search and recommendation
• gradient conflict
• parameter-efficient fine-tuning
• general-domain knowledge
• LLMs

Innovation

• Gradient Multi-Subspace Tuning
• Parameter-Efficient Fine-Tuning
• Search and Recommendation Unification
• Null-Space Projection
• Multi-Subspace Decomposition