How to Interpret Agent Behavior

2026-05-13

AI Summary

This work addresses the challenge of interpreting and analyzing long-horizon autonomous agent behaviors, which are typically recorded as unstructured natural language traces that hinder efficient diagnosis of inefficiencies, errors, and compliance violations. To overcome this limitation, the authors introduce ACT*ONOMY—the first grounded theory–based behavioral taxonomy for autonomous agents—featuring a three-level hierarchical structure comprising 10 actions, 46 sub-actions, and 120 leaf categories. Accompanied by an open knowledge base and an automated annotation pipeline, ACT*ONOMY provides a scalable, shared semantic vocabulary and standardized analysis protocol. This framework enables systematic cross-agent behavior comparison, facilitates failure mode identification, and reveals behavior patterns correlated with system faults, thereby substantially enhancing human oversight and control over autonomous agents.

Abstract

Autonomous agents such as Claude Code and Codex now operate for hours or even days. Understanding their runtime behavior has become critical for downstream tasks such as diagnosing inefficiencies, fixing bugs, and ensuring better oversight. A primary way to gain this understanding is analyzing the reasoning trajectories and execution traces these agents generate. Yet such data remains in unstructured natural-language form, making it difficult for humans to interpret at scale. We introduce ACT*ONOMY (a combination of Action and Taxonomy), a taxonomy for describing and analyzing agent behavior at runtime. ACT*ONOMY has two components: (1) the taxonomy itself, developed through Grounded Theory and structured as a three-level hierarchy of 10 actions, 46 subactions, and 120 leaf categories; and (2) an open repository that hosts the living taxonomy, provides an automated analysis pipeline that applies it to agent trajectories analysis, and defines an extension protocol for customization and growth. Our experiments show that ACTONOMY can compare behavioral profiles across agents and characterize a single agent's behavior across diverse trajectories, surfacing patterns indicative of failure modes. By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently, enabling better oversight and control.

Problem

• agent behavior
• reasoning trajectories
• execution traces
• interpretability
• runtime analysis

Innovation

• agent behavior interpretation
• behavior taxonomy
• reasoning trajectories
• structured analysis
• autonomous agents