Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations

2026-04-20

AI Summary

This work addresses the substantial storage and computational overhead incurred by high-dimensional representations generated by large language models (LLMs) in recommender systems, a challenge exacerbated by existing compression methods that rely on suboptimal final-layer embeddings. The study is the first to identify and theoretically explain the phenomenon—termed Middle-layer Representation Advantage (MRA)—where intermediate LLM representations outperform final-layer outputs for recommendation tasks. Building on modularity theory, the authors propose MARC, a framework that explicitly orchestrates internal functional specialization within LLMs through modular adaptation (introducing dedicated compression and task-adaptation modules) and modular task decoupling (integrating information-theoretic constraints with heterogeneous architectures). Extensive online A/B testing in a large-scale commercial search advertising system demonstrates the framework’s efficacy, yielding a statistically significant 2.82% increase in eCPM, thereby validating its practical utility and effectiveness.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recently, large language models (LLMs) have advanced recommendation systems (RSs), and recent works have begun to explore how to integrate LLMs into industrial RSs. While most approaches deploy LLMs offline to generate and pre-cache augmented representations for RSs, high-dimensional representations from LLMs introduce substantial storage and computational costs. Thus, it is crucial to compress LLM representations effectively. However, we identify a counterintuitive phenomenon during representation compression: Mid-layer Representation Advantage (MRA), where representations from middle layers of LLMs outperform those from final layers in recommendation tasks. This degraded final layer renders existing compression methods, which typically compress on the final layer, suboptimal. We interpret this based on modularity theory that LLMs develop spontaneous internal functional modularity and force the final layer to specialize in the proxy training task. Thus, we propose \underline{M}odul\underline{a}r \underline{R}epresentation \underline{C}ompression (MARC) to explicitly control the modularity of LLMs. First, Modular Adjustment explicitly introduces compression and task adaptation modules, enabling the LLM to operate strictly as a representation-learning module. Next, to ground each module to its specific task, Modular Task Decoupling uses information constraints and different network structures to decouple tasks. Extensive experiments validate that MARC addresses MRA and produces efficient representations. Notably, MARC achieved a 2.82% eCPM lift in an online A/B test within a large-scale commercial search advertising scenario.

Problem

• representation compression
• large language models
• recommendation systems
• modularity
• mid-layer representation advantage

Innovation

• Modular Representation Compression
• Mid-layer Representation Advantage
• Task Decoupling
• Large Language Models
• Recommendation Systems