SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs

2026-09-09

AI Summary

本文通过引入SearchAtlas框架，将搜索轨迹转换为结构化图来分析代理搜索策略，揭示证据传播过程，解决评估过程中忽视搜索过程的问题。

Abstract

LLM search agents are often evaluated on final-answer accuracy, overlooking the process. Analyzing a search strategy requires understanding how credible evidence is retrieved to address question constraints. This valuable information is buried in raw search trajectories that are long and difficult to parse. We introduce SearchAtlas, a framework that converts search trajectories into structured graphs whose edges represent how evidence is propagated across the reasoning trace, from the query that retrieves it to the final answer. Our automated parsing pipeline achieves a mean edge F1 of 86.0% against human-annotated graphs and remains consistent across repeated runs. We analyze five search agents on three benchmarks, revealing systematic differences in search scale and evidence aggregation. SearchAtlas exposes fragmented answer support, question constraints that do not reach the answer, and unverified parametric knowledge entering the response. These process failures are strongly associated with incorrect answers, even more so than an LLM judge given either the raw trajectory or the ordered query list, suggesting that the constructed graphs provide useful interpretability. Moreover, an audit of cases in which process-diagnostic scores disagree with final-answer correctness shows that they capture information not reducible to answer accuracy.

Problem

• LLM search agents
• search strategy
• evidential query graphs
• process analysis
• final-answer accuracy

Innovation

• SearchAtlas
• evidential query graphs
• search trajectories
• process analysis
• information propagation