WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent

2025-08-07

AI Summary

Existing deep research agents predominantly rely on textual inputs, limiting their capability to perform vision-language collaborative reasoning required for complex real-world information retrieval tasks. Method: We propose BrowseAgent, the first intelligent agent framework explicitly designed for vision-language deep research, integrating multimodal perception, logical reasoning, and tool-augmented execution. To enable cold-start training, we curate high-quality synthetic trajectory data; we further introduce BrowseComp-VL, a dedicated benchmark for evaluating cross-modal retrieval performance. Our approach synergistically combines vision-language models, reinforcement learning, and synthetic-data-driven training. Contribution/Results: Extensive experiments demonstrate that BrowseAgent significantly outperforms closed-source baselines, retrieval-augmented generation (RAG) pipelines, and state-of-the-art open-source agents across four challenging vision-question answering benchmarks, validating its superiority and practicality in multimodal information-seeking tasks.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Web agents such as Deep Research have demonstrated superhuman cognitive abilities, capable of solving highly challenging information-seeking problems. However, most research remains primarily text-centric, overlooking visual information in the real world. This makes multimodal Deep Research highly challenging, as such agents require much stronger reasoning abilities in perception, logic, knowledge, and the use of more sophisticated tools compared to text-based agents. To address this limitation, we introduce WebWatcher, a multi-modal Agent for Deep Research equipped with enhanced visual-language reasoning capabilities. It leverages high-quality synthetic multimodal trajectories for efficient cold start training, utilizes various tools for deep reasoning, and further enhances generalization through reinforcement learning. To better evaluate the capabilities of multimodal agents, we propose BrowseComp-VL, a benchmark with BrowseComp-style that requires complex information retrieval involving both visual and textual information. Experimental results show that WebWatcher significantly outperforms proprietary baseline, RAG workflow and open-source agents in four challenging VQA benchmarks, which paves the way for solving complex multimodal information-seeking tasks.

Problem

• Enhancing visual-language reasoning in deep research agents
• Addressing limitations of text-centric multimodal research approaches
• Improving performance in complex visual question answering tasks

Innovation

• Enhanced visual-language reasoning capabilities
• Synthetic multimodal trajectories for training
• Reinforcement learning for generalization