Auditable Agents

2026-04-07

AI Summary

This study addresses the critical lack of accountability in large language model (LLM) agents following external actions, which hinders traceability and responsibility attribution. The work introduces the first systematic framework for agent auditability, articulating five core dimensions and proposing an “Auditability Card” to standardize assessment. It further develops a full-lifecycle auditing architecture integrating detection, enforcement, and recovery mechanisms. Leveraging runtime intervention, tamper-proof logging, and log-recovery techniques—validated through ecosystem-wide security evaluations and controlled experiments—the study identifies 617 security flaws across mainstream open-source LLM agent projects. Experimental results demonstrate that a pre-execution mediation layer incurs only 8.3 milliseconds of overhead and that partial reconstruction of accountability-critical information remains feasible even in the absence of complete logs.

Abstract

LLM agents call tools, query databases, delegate tasks, and trigger external side effects. Once an agent system can act in the world, the question is no longer only whether harmful actions can be prevented--it is whether those actions remain answerable after deployment. We distinguish accountability (the ability to determine compliance and assign responsibility), auditability (the system property that makes accountability possible), and auditing (the process of reconstructing behavior from trustworthy evidence). Our claim is direct: no agent system can be accountable without auditability. To make this operational, we define five dimensions of agent auditability, i.e., action recoverability, lifecycle coverage, policy checkability, responsibility attribution, and evidence integrity, and identify three mechanism classes (detect, enforce, recover) whose temporal information-and-intervention constraints explain why, in practice, no single approach suffices. We support the position with layered evidence rather than a single benchmark: lower-bound ecosystem measurements suggest that even basic security prerequisites for auditability are widely unmet (617 security findings across six prominent open-source projects); runtime feasibility results show that pre-execution mediation with tamper-evident records adds only 8.3 ms median overhead; and controlled recovery experiments show that responsibility-relevant information can be partially recovered even when conventional logs are missing. We propose an Auditability Card for agent systems and identify six open research problems organized by mechanism class.

Problem

• auditability
• accountability
• LLM agents
• responsibility attribution
• evidence integrity

Innovation

• auditability
• LLM agents
• accountability
• tamper-evident logging
• responsibility attribution