Uncovering Systemic and Environment Errors in Autonomous Systems Using Differential Testing

2025-07-04

AI Summary

Attributing task failures in autonomous agents is challenging—distinguishing between intrinsic agent deficiencies (e.g., flawed models or policies) and extrinsic environmental constraints that render tasks inherently infeasible remains difficult. Method: We propose AIProbe, the first differential testing framework tailored for black-box agents, enabling systematic failure attribution. It employs Latin hypercube sampling to generate diverse environment configurations and integrates a search-based planner to independently solve each task; discrepancies between planner and agent behaviors are then analyzed for differential diagnosis. Contribution/Results: Across multiple domains, AIProbe significantly improves failure detection rates—outperforming state-of-the-art approaches both in total error identification and unique error discovery—thereby enhancing the reliability and deployability of autonomous agents.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Multiagent Systems: Adversarial Agents
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

When an autonomous agent behaves undesirably, including failure to complete a task, it can be difficult to determine whether the behavior is due to a systemic agent error, such as flaws in the model or policy, or an environment error, where a task is inherently infeasible under a given environment configuration, even for an ideal agent. As agents and their environments grow more complex, identifying the error source becomes increasingly difficult but critical for reliable deployment. We introduce AIProbe, a novel black-box testing technique that applies differential testing to attribute undesirable agent behaviors either to agent deficiencies, such as modeling or training flaws, or due to environmental infeasibility. AIProbe first generates diverse environmental configurations and tasks for testing the agent, by modifying configurable parameters using Latin Hypercube sampling. It then solves each generated task using a search-based planner, independent of the agent. By comparing the agent's performance to the planner's solution, AIProbe identifies whether failures are due to errors in the agent's model or policy, or due to unsolvable task conditions. Our evaluation across multiple domains shows that AIProbe significantly outperforms state-of-the-art techniques in detecting both total and unique errors, thereby contributing to a reliable deployment of autonomous agents.

Problem

• Identify systemic vs environment errors in autonomous systems
• Differentiate agent deficiencies from environmental infeasibility
• Improve error detection for reliable autonomous agent deployment

Innovation

• Uses differential testing for error attribution
• Generates diverse environments via Latin Hypercube sampling
• Compares agent performance with search-based planner