AgentXRay: White-Boxing Agentic Systems via Workflow Reconstruction

2026-02-05

AI Summary

This work addresses the challenge of interpreting and controlling black-box agent systems, whose internal workflows are opaque. To this end, it introduces the task of *agent workflow reconstruction*, which aims to rebuild black-box behaviors into interpretable and editable explicit white-box workflows using only input–output access. The problem is formally framed as a combinatorial optimization task, and a Monte Carlo Tree Search–based framework is proposed, augmented with a score-driven red-black pruning mechanism to efficiently explore sequences of roles and tool calls within a chain-structured workflow space. Experiments demonstrate that, compared to unpruned baselines, the proposed method significantly improves behavioral fidelity across multiple domains while reducing token consumption, thereby enabling deeper exploration of high-quality workflows under a fixed iteration budget.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Multiagent Systems: Teamwork
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models have shown strong capabilities in complex problem solving, yet many agentic systems remain difficult to interpret and control due to opaque internal workflows. While some frameworks offer explicit architectures for collaboration, many deployed agentic systems operate as black boxes to users. We address this by introducing Agentic Workflow Reconstruction (AWR), a new task aiming to synthesize an explicit, interpretable stand-in workflow that approximates a black-box system using only input--output access. We propose AgentXRay, a search-based framework that formulates AWR as a combinatorial optimization problem over discrete agent roles and tool invocations in a chain-structured workflow space. Unlike model distillation, AgentXRay produces editable white-box workflows that match target outputs under an observable, output-based proxy metric, without accessing model parameters. To navigate the vast search space, AgentXRay employs Monte Carlo Tree Search enhanced by a scoring-based Red-Black Pruning mechanism, which dynamically integrates proxy quality with search depth. Experiments across diverse domains demonstrate that AgentXRay achieves higher proxy similarity and reduces token consumption compared to unpruned search, enabling deeper workflow exploration under fixed iteration budgets.

Problem

• agentic systems
• interpretability
• black-box
• workflow reconstruction
• white-boxing

Innovation

• Agentic Workflow Reconstruction
• White-box Interpretability
• Monte Carlo Tree Search
• Red-Black Pruning
• Chain-structured Workflow