Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation

2026-05-28

AI Summary

This work addresses the challenges in open-ended multimodal deep research—specifically, the absence of definitive ground truth and the difficulty of aligning textual and visual evidence—by introducing Ptah, the first multi-agent framework that supports interleaved text-and-image reasoning with end-to-end verifiability. Ptah integrates planning, research, and writing phases through declarative multimodal tool invocation, a visual working memory mechanism, and dedicated verification agents to ensure factual accuracy, precise citation, and cross-modal consistency in generated reports. The accompanying PtahEval evaluation protocol demonstrates Ptah’s significant advantages on deep research benchmarks, showing superior performance over strong baselines in reliability, visual informativeness, and human usability.

Abstract

Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep research, which synthesizes scattered evidence into long-form reports. However, verifiable multimodal deep research remains challenging due to open-ended synthesis without deterministic ground truth and the need to interleave textual arguments with visual evidence. We propose \textsc{Ptah}, a multi-agent harness for interleaved report generation. \textsc{Ptah} orchestrates the lifecycle from user query to rendered web report through planning, research, and writing stages, where specialized agents construct visual-aware plans, collect claim-grounded evidence, maintain source-aligned images in a \textit{Visual Working Memory}, and compose reports through declarative multimodal tool use. A verifier agent serves as the harness's acceptance function, enforcing factual grounding, citation fidelity, and cross-modal consistency throughout the workflow. We further introduce \textsc{Ptah}Eval, an evaluation protocol that augments existing benchmarks with image-level and presentation-level assessments. Experiments on deep research benchmarks show that \textsc{Ptah} produces more reliable, visually informative, and usable human-facing multimodal reports than strong baselines.

Problem

• verifiable multimodal deep research
• interleaved report generation
• factual grounding
• cross-modal consistency
• visual evidence

Innovation

• multimodal deep research
• multi-agent system
• visual working memory
• verifiable report generation
• cross-modal consistency