Interactive Debugging and Steering of Multi-Agent AI Systems

2025-03-03

AI Summary

To address core challenges in developing LLM-powered multi-agent systems—including difficult error localization, lack of interactive debugging support, and inefficient configuration iteration—this paper introduces AGDebugger, the first visual debugging tool enabling message-level editing and agent-state resetting. Methodologically, it proposes: (1) an interactive, message-level debugging paradigm supporting real-time intervention, message rewriting, and selective agent state reset; (2) an overview-detail timeline visualization for navigating multi-agent dialogue history; and (3) a two-phase user study with 14 participants to empirically evaluate usability and efficacy. Results show AGDebugger significantly improves error localization efficiency (reducing time by 47% on average) and accelerates configuration iteration. Furthermore, the study yields actionable, reusable interface design principles for multi-agent system debugging.

Technology Category

• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Fully autonomous teams of LLM-powered AI agents are emerging that collaborate to perform complex tasks for users. What challenges do developers face when trying to build and debug these AI agent teams? In formative interviews with five AI agent developers, we identify core challenges: difficulty reviewing long agent conversations to localize errors, lack of support in current tools for interactive debugging, and the need for tool support to iterate on agent configuration. Based on these needs, we developed an interactive multi-agent debugging tool, AGDebugger, with a UI for browsing and sending messages, the ability to edit and reset prior agent messages, and an overview visualization for navigating complex message histories. In a two-part user study with 14 participants, we identify common user strategies for steering agents and highlight the importance of interactive message resets for debugging. Our studies deepen understanding of interfaces for debugging increasingly important agentic workflows.

Problem

• Challenges in debugging long AI agent conversations
• Lack of interactive debugging tools for multi-agent systems
• Need for iterative agent configuration and message editing

Innovation

• Interactive UI for multi-agent debugging
• Edit and reset prior agent messages
• Visualization for navigating message histories