ParaRecover: A Process-Level Benchmark for Error Localization and Recovery in Parallel Tool-Use Agents

2026-09-10

AI Summary

ParaRecover通过构建包含14种错误类型的基准测试，评估并改进多轮并行工具使用代理在错误定位和恢复方面的能力。

Abstract

Existing agent benchmarks mainly evaluate final task success or tool-call correctness, providing limited insight into whether agents can reliably diagnose and recover from intermediate execution failures. This limitation becomes particularly critical in multi-turn parallel tool-use scenarios, where errors may propagate across dependent branches and trigger cascading failures. We introduce ParaRecover, a process-level benchmark for evaluating error localization and recovery in multi-turn parallel tool-use agents. Built upon a fine-grained taxonomy of 14 error types covering planning dependencies, tool selection, and argument matching, the benchmark comprises 10,626 instances spanning two difficulty levels. To enable finegrained, process-oriented evaluation, we further propose the SDE rubric, which measures structural integrity, diagnostic reasoning, and evolutionary strategy during agent execution.Experiments across more than ten mainstream LLMs reveal that even state-of-the-art models still struggle with multi-turn error propagation,implicit tool-use failures, and precise replanning. Moreover, we demonstrate that the SDE rubric provides effective supervision signals for improving agents' reflective recovery capabilities. Our data and code are available at https://github.com/gbw206/ParaRecover.

Problem

• error localization
• recovery
• parallel tool-use agents
• multi-turn scenarios
• cascading failures

Innovation

• ParaRecover
• error localization and recovery
• multi-turn parallel tool-use
• SDE rubric
• reflective recovery capabilities