The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit

2025-01-04

AI Summary

To address the efficiency–accuracy trade-off in click-through rate (CTR) prediction for large language model (LLM)-based recommender systems, this paper proposes a lightweight and efficient framework integrating retrieval-augmented generation (RAG) with a multi-head early-exit mechanism. We introduce a confidence-driven multi-head early-exit architecture that enables dynamic inference termination based on per-head output certainty. Additionally, we incorporate a lightweight graph convolutional network (GCN) as the retriever within the RAG pipeline, enabling joint optimization of retrieval and generation. Extensive experiments on mainstream CTR benchmark datasets demonstrate that our method reduces average inference latency by 42.3% while consistently improving AUC by 0.12–0.31 percentage points. The framework thus satisfies industrial requirements for real-time recommendation—delivering both low-latency inference and high predictive accuracy.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture to concurrently enhance both aspects. By integrating Graph Convolutional Networks (GCNs) as efficient retrieval mechanisms, we are able to significantly reduce data retrieval times while maintaining high model performance. The early exit strategy employed allows for dynamic termination of model inference, utilizing real-time predictive confidence assessments across multiple heads. This not only quickens the responsiveness of LLMs but also upholds or improves their accuracy, making it ideal for real-time application scenarios. Our experiments demonstrate how this architecture effectively decreases computation time without sacrificing the accuracy needed for reliable recommendation delivery, establishing a new standard for efficient, real-time LLM deployment in commercial systems.

Problem

• Recommendation Systems
• Large Language Models
• Speed-Accuracy Tradeoff

Innovation

• Graph Convolutional Network
• Enhanced Retrieval
• Multi-channel Rapid Decision-making