Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization

2024-10-13

arXiv.org

AI Summary

This paper addresses the challenge of heterogeneous retrieval requirements across multi-task RAG agents by proposing the first unified search engine supporting joint offline learning and online adaptation. Methodologically, it introduces an iterative utility maximization algorithm grounded in the Expectation-Maximization (EM) framework, which jointly models multi-agent feedback to optimize personalized retrieval ranking (Learning to Rank). Key contributions include: (1) the first application of the EM framework to RAG retrieval optimization, enabling joint calibration across heterogeneous LLMs and RAG strategies; and (2) a novel offline–online co-adaptation mechanism that balances system stability with real-time responsiveness. Evaluated on 18 RAG models from the KILT benchmark, our approach achieves statistically significant improvements over strong baselines. Ablation studies validate the efficacy of each component and confirm consistent, agent-specific retrieval gains.

Technology Category

• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper investigates the design of a unified search engine to serve multiple retrieval-augmented generation (RAG) agents, each with a distinct task, backbone large language model (LLM), and retrieval-augmentation strategy. We introduce an iterative approach where the search engine generates retrieval results for these RAG agents and gathers feedback on the quality of the retrieved documents during an offline phase. This feedback is then used to iteratively optimize the search engine using a novel expectation-maximization algorithm, with the goal of maximizing each agent's utility function. Additionally, we adapt this approach to an online setting, allowing the search engine to refine its behavior based on real-time individual agents feedback to better serve the results for each of them. Experiments on diverse datasets from the Knowledge-Intensive Language Tasks (KILT) benchmark demonstrates that our approach significantly on average outperforms competitive baselines across 18 RAG models. We also demonstrate that our method effectively ``personalizes'' the retrieval process for each RAG agent based on the collected feedback. Finally, we provide a comprehensive ablation study to explore various aspects of our method.

Problem

• Designing a unified search engine for multiple RAG agents
• Optimizing retrieval results via iterative feedback and utility maximization
• Personalizing retrieval for each agent based on real-time feedback

Innovation

• Unified search engine for multiple RAG agents
• Iterative optimization using feedback and EM algorithm
• Online adaptation with real-time agent feedback