Graph of Trace: Visualizing Execution Traces of Scientific Agent

2026-06-13

AI Summary

This work addresses the lack of transparency in scientific AI agents when executing complex research workflows, which hinders human understanding, scrutiny, and intervention. To bridge this gap, the paper introduces the first monitoring and visualization framework that dynamically models an agent’s execution trace as a structured directed graph in real time. By capturing fine-grained intermediate events—such as tool invocations and code executions—the framework renders the evolving workflow structure explicitly. This approach substantially enhances the interpretability and controllability of AI-driven scientific processes. Empirical evaluations across AI, neuroscience, and biology demonstrate strong endorsement from domain experts, confirming its effectiveness in supporting result traceability, fault localization, and human–agent collaborative analysis.

Abstract

Scientific AI agents can autonomously carry out complex research workflows, yet these unfolded workflows often remain difficult for humans to inspect and review, limiting interpretable, controllable and effective human-AI collaboration. To address this challenge, we present a monitoring and visualization framework that records fine-grained execution events and organizes them into a directed graph that makes agent workflows explicit as they proceed. The system records intermediate steps (e.g. tool calls and code executions), and renders them as real-time updated visual traces that expose workflow structure. This allows users to examine how results are produced, identify where failures emerge, and better understand agent behavior across different stages of the research process. We conduct an evaluation on complex research tasks with domain experts of interdisciplinary backgrounds in AI, neuroscience, and biology. Experts report that structured traces visualization improves understanding of agent workflows, perceived interpretability, and usability for analysis and further interaction.

Problem

• scientific AI agents
• execution traces
• workflow interpretability
• human-AI collaboration
• visualization

Innovation

• execution trace visualization
• scientific AI agents
• workflow interpretability
• directed graph representation
• human-AI collaboration