MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains

2026-02-28

AI Summary

Existing benchmarks struggle to evaluate multimodal agents’ adaptive planning and cross-modal reasoning capabilities over long reasoning chains. To address this gap, this work proposes MC-Search, the first benchmark tailored for multimodal retrieval-augmented generation (MM-RAG), comprising 3,333 structured multi-hop reasoning samples annotated with five novel reasoning structure types. The authors introduce the HAVE mechanism to ensure traceability of evidence and develop the Search-Align framework to enable model optimization under process supervision. Experiments reveal systematic deficiencies in mainstream multimodal large language models regarding retrieval and planning, while Search-Align substantially improves both planning accuracy and retrieval fidelity in open-source models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Metareasoning and Metaheuristics
• Multiagent Systems: Multiagent Planning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

With the increasing demand for step-wise, cross-modal, and knowledge-grounded reasoning, multimodal large language models (MLLMs) are evolving beyond the traditional fixed retrieve-then-generate paradigm toward more sophisticated agentic multimodal retrieval-augmented generation (MM-RAG). Existing benchmarks, however, mainly focus on simplified QA with short retrieval chains, leaving adaptive planning and multimodal reasoning underexplored. We present MC-Search, the first benchmark for agentic MM-RAG with long, step-wise annotated reasoning chains spanning five representative reasoning structures. Each example specifies sub-questions, retrieval modalities, supporting facts, and intermediate answers, with fidelity ensured by HAVE (Hop-wise Attribution and Verification of Evidence), resulting in 3,333 high-quality examples averaging 3.7 hops. Beyond answer accuracy, MC-Search introduces new process-level metrics for reasoning quality, stepwise retrieval and planning accuracy. By developing a unified agentic MM-RAG pipeline, we benchmark six leading MLLMs and reveal systematic issues such as over- and under-retrieval and modality-misaligned planning. Finally, we introduce Search-Align, a process-supervised fine-tuning framework leveraging verified reasoning chains, showing that our data not only enables faithful evaluation but also improves planning and retrieval fidelity in open-source MLLMs.

Problem

• multimodal reasoning
• agentic search
• long reasoning chains
• retrieval-augmented generation
• adaptive planning

Innovation

• agentic MM-RAG
• structured reasoning chains
• multimodal retrieval
• process-level evaluation
• Search-Align