From Reaction to Anticipation: Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation

2026-05-12

AI Summary

This work addresses the vulnerability of robotic manipulation tasks to failures in dynamic, unstructured environments, where existing approaches rely on post-hoc detection and reactive recovery, resulting in high latency and insufficient robustness. To overcome these limitations, the authors propose AgentChord, a novel multi-agent system that incorporates human-inspired prospective planning into robotic fault recovery for the first time. AgentChord structures the recovery process around three specialized agents—Composer, Orchestrator, and Conductor—and models tasks as directed task graphs with pre-embedded, context-aware recovery branches. By integrating low-latency monitoring with precompiled recovery strategies, the system enables proactive anticipation and immediate response without requiring online replanning. Experimental results demonstrate that AgentChord significantly improves success rates and execution efficiency across diverse long-horizon bimanual manipulation tasks, thereby enhancing robotic reliability and autonomy in real-world settings.

Abstract

Although robotic manipulation has made significant progress, reliable execution remains challenging because task failures are inevitable in dynamic and unstructured environments. To handle such failures, existing frameworks typically follow a stepwise detect-reason-recover pipeline, which often incurs high latency and limited robustness due to delayed reasoning and reactive planning. Inspired by the human capability to anticipate and proactively plan for potential failures, we introduce AgentChord, an agentic system that models a manipulation task as a directed task graph. Before execution, this graph is enriched with anticipatory recovery branches that specify context-aware corrective behaviors, enabling immediate and targeted responses when failures occur. Specifically, AgentChord operates through a choreography of specialized agents: a composer that structures the nominal task graph, an arranger that augments the graph with anticipatory recovery branches, and a conductor that compiles and coordinates executable transitions using low-latency monitors to detect deviations and trigger pre-compiled recoveries without re-planning. Empirical studies on diverse long-horizon bimanual manipulation tasks demonstrate that AgentChord substantially improves success rates and execution efficiency, advancing the reliability and autonomy of real-world robotic systems. The project page is available at: https://shengxu.net/AgentChord/.

Problem

• robotic manipulation
• failure recovery
• proactive planning
• task graph
• reactive systems

Innovation

• proactive failure recovery
• agentic task graph
• anticipatory planning
• robotic manipulation
• multi-agent coordination