Automata from Agent Traces: Failure and Next-Step Prediction

2026-08-24

AI Summary

为解决LLM代理行为结构不透明问题，本文提出将多条轨迹压缩成一个紧凑的有限状态机(FSM)，用于预测下一步行动和失败。

Abstract

LLM-based agents execute multi-step tasks, but their behavioral structure remains opaque: long unstructured traces resist the safety auditing and runtime monitoring that deployment requires. Existing approaches operate per-trace or success-only, so they miss the cross-run topology that links next-step and failure prediction. To recover that shared structure, we collapse an entire trace corpus into a single, compact finite-state machine (FSM) that serves as a structural substrate for the otherwise unpredictable behavior of LLM agents. Across twelve public datasets, the FSMs are compact (7-43 states), replay held-out data at >=0.997 fitness with near-identical topology across splits, and build in milliseconds. This substrate addresses both prediction goals. For next-step prediction, FSM-state context outperforms Agent Workflow Memory on every ground-truth-matched dataset. For failure prediction, per-state behavioral features reach held-out AUROC up to 0.94, and an online monitor ranks failing runs above passing ones from a partial trace, triggering early stopping well before completion. Behavioral topology thus appears shaped more by the deployment harness than by the LLM, providing a model-agnostic structural primitive for safety auditing and runtime monitoring.

Problem

• Automata
• Agent Traces
• Failure Prediction
• Next-Step Prediction
• Behavioral Structure

Innovation

• finite-state machine
• next-step prediction
• failure prediction
• behavioral structure
• safety auditing