To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAG

2026-06-23

AI Summary

This work addresses the high computational cost of document evaluation in multi-agent retrieval-augmented generation and the unclear evaluation mechanisms of smaller models. Through training-free intervention, the study systematically analyzes the evaluation behavior of 7B–9B instruction-tuned models in multi-document question answering, revealing that weaker models primarily rely on document isolation, whereas stronger models depend more on quality scoring. Building on these insights, the authors propose Reasoning-Score Coupling—a label-free probing method—and MADARA, a model-adaptive routing architecture. Experiments show that document isolation alone enables weak models to match the performance of full multi-agent evaluation, yielding gains of up to 50 percentage points. Moreover, MADARA substantially reduces computational overhead, and its diagnostic threshold generalizes zero-shot across four unseen model families, enabling efficient and lightweight cross-model transfer.

Abstract

Multi-agent document assessment for retrieval-augmented generation is computationally expensive, driving practitioners toward smaller, deployable models whose assessment mechanisms remain poorly understood. We conduct a controlled study of training-free interventions on 7B-9B instruction-tuned models across diverse QA benchmarks, revealing a sharp dichotomy in how models benefit from assessment. For weaker baselines, the dominant mechanism is per-document isolation. Astoundingly, assessment-free isolation matches full multi-agent assessment, demonstrating that resolving multi-document context confusion, rather than scoring quality, drives outsized gains of up to 50 percentage points. Conversely, for strong baselines where scoring quality matters, we introduce Reasoning-Score Coupling, a label-free perturbation probe that classifies scoring behavior. Integrating these findings, we propose MADARA, a model-adaptive routing architecture. Crucially, MADARA's diagnostic thresholds derived from a single pilot model generalize zero-shot to four unseen model families, providing a robust, lightweight pipeline to eliminate computational overhead.

Problem

• multi-agent RAG
• document assessment
• computational efficiency
• model adaptivity
• retrieval-augmented generation

Innovation

• model-adaptive routing
• document isolation
• reasoning-score coupling
• training-free intervention
• multi-agent RAG