Runtime Compliance Verification for AI Agents

2026-06-17

AI Summary

This work addresses the challenge of ensuring real-time GDPR compliance for AI agents that process personal data during execution, a scenario where traditional offline testing falls short. The authors propose C-Trace, a novel framework that enables runtime verification of GDPR compliance by formalizing core principles—such as consent, purpose limitation, data minimization, and the right to erasure—into policy predicates. A runtime monitor dynamically intercepts non-compliant tool invocations and model outputs based on these predicates. Integrated with noise-robust information extraction and evaluated under adversarial dialogue attacks—including DSPy-based prompts and red-team corpora—the approach achieves ≤12% attack success rate and ≤16% false positive rate across four GDPR-relevant scenarios, even with 10% extraction noise; under ideal conditions, it attains 0% attack success.

Abstract

AI agents now handle personal data through tool use, function calls, and multi turn dialogue, which can create obligations under the General Data Protection Regulation (GDPR). Current testing practices mainly rely on offline red teaming or static prompt review, but they do not guarantee at runtime that agent behavior follows regulatory rules. We propose C-Trace (Compliance Trace based Runtime Agent Conformance Enforcement), a verification framework that: (i) expresses a subset of GDPR requirements, including consent, purpose limitation, data minimization, and the right to erasure, as formal policy predicates over agent execution traces; (ii) uses a runtime monitor that intercepts every tool invocation and model output and rejects non-compliant actions; and (iii) tests the agent with attack dialogues, including DSPy generated prompts and verbatim prompts from red teaming corpora, that try to induce violations. We evaluate the framework on four case studies reframed to GDPR. Under 10 percent per-category extractor noise, including drop-out and over-typing, the monitor keeps the attack success rate at less than or equal to 12 percent, below the baselines we compare against, and false positives at less than or equal to 16 percent, and reaches 0 percent ASR under perfect extraction.

Problem

• Runtime Compliance
• AI Agents
• GDPR
• Data Protection
• Regulatory Verification

Innovation

• runtime compliance verification
• GDPR enforcement
• policy predicates
• execution trace monitoring
• AI agent security