DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment

2026-07-08

AI Summary

Existing tool-augmented agents struggle to continuously evolve through their own experience: supervised fine-tuning relies on fixed trajectories, while sparse-reward reinforcement learning suffers from insufficient supervision in long-horizon tasks. This work proposes DeepSearch-Evolve, a self-distillation framework coupled with DeepSearch-World—a verifiable, deterministic environment—to establish an iterative evolution loop encompassing trajectory generation, filtering, mixing, and fine-tuning. It introduces the first reproducible web-search environment featuring progress validation, fact-based reflection, and failure recovery mechanisms. Notably, the agent achieves self-improvement without requiring a stronger teacher model. Experiments show that DeepSearch-World-9B attains performance of 31.2%, 61.5%, and 93.4% on BrowseComp, GAIA, and HotpotQA benchmarks, respectively, matching the capabilities of current state-of-the-art open-source agents.

Abstract

Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-horizon interactions. We present DeepSearch-Evolve, a self-distillation framework for web agents built on DeepSearch-World, a deterministic and verifiable environment with reproducible search and page-reading tools. DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks and supports key agentic cognitive behaviors useful for self-evolving, including progress verification, grounded reflection, and failure recovery. DeepSearch-Evolve iteratively performs trajectory generation, filtering, data mixing, and fine-tuning to train stronger agents. Without distillation from more capable models, DeepSearch-World-9B achieves competitive performance compared with open-source agents, reaching 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA, showing that verifiable environments enable scalable self-evolution for long-horizon web agents. We will release the environment, 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.

Problem

• self-distillation
• tool-use agents
• long-horizon interactions
• verifiable environment
• deep search agents

Innovation

• self-distillation
• verifiable environment
• tool-use agents
• long-horizon reasoning
• agent self-evolution