A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents

2026-09-15

AI Summary

研究分析了157个开源LLM代理项目的质量保证实践，发现当前QA主要关注基本功能和高风险操作，覆盖范围碎片化，需要系统性的端到端验证以确保代理工作流程的安全性和可靠性。

Abstract

Large language model (LLM)-based agents are increasingly used across software engineering, web automation, research, and productivity applications. Their integration of planning, memory, tool use, code execution, and external interactions enables greater autonomy but also introduces new reliability, safety, and security risks. We present a large-scale empirical study of quality assurance (QA) practices in 157 open-source LLM-based agent projects with at least 100 GitHub stars. We analyze documentation, source code, configurations, and tests to characterize QA practices across execution surfaces, safeguards, testing artifacts, risk scenarios, and recurring gaps. We find that current QA primarily focuses on basic functionality and high-risk actions, while coverage remains fragmented. Safeguards are inconsistently applied across equivalent execution routes, tests rarely examine boundary, adversarial, or multi-step tool-use failures, and identified risks are seldom translated into end-to-end QA checks. These findings highlight the need to move beyond feature-level testing toward systematic end-to-end validation that ensures agent workflows remain within intended boundaries when interacting with untrusted inputs, tools, persistent state, and external APIs.

Problem

• Quality Assurance
• Large Language Models
• Reliability
• Safety
• Security

Innovation

• Quality Assurance
• Large Language Models
• End-to-End Validation
• Systematic Testing
• Security Risks