SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation

2026-07-09

AI Summary

This work addresses the challenge of insufficient test coverage in automated test generation caused by “lazy” behaviors of code-generating agents—such as premature termination or avoidance of complex logic—and the inefficiency of existing human-in-the-loop supervision. To overcome these limitations, the authors propose SCATE, a novel framework that formulates the supervision of code-generating agents as a contextual multi-armed bandit problem. SCATE dynamically selects optimal testing actions by leveraging real-time code coverage and class testability metrics, enabling fully automated, adaptive oversight without manual intervention. Experimental results demonstrate that SCATE improves line coverage by 32.3% and branch coverage by 30.9% on the GEMINI-CLI benchmark, significantly outperforming baseline approaches. Furthermore, the framework exhibits strong generalizability and adaptability across diverse large language model–based agents, including Claude Code.

Abstract

While autonomous coding agents have significantly advanced automated test generation, they remain fundamentally limited by lazy generation, a phenomenon where agents prematurely terminate tasks and systematically avoid complex programmatic logic, resulting in inadequate code coverage. Currently, mitigating this premature termination requires continuous human-in-the-loop supervision. This heavy reliance on human intuition creates a bottleneck that negates the efficiency gains of automated generation. We propose SCATE, a framework for adaptive, automated supervision of coding agents that replaces human intervention during test generation. By formulating supervision as a contextual bandit problem, SCATE learns to select the most promising testing actions based on the current coverage and class testability metrics, maximizing coverage gains while minimizing wasted generation effort. Our empirical evaluation demonstrates that SCATE integrates seamlessly with different coding agents. When applied to GEMINI-CLI, it achieves 32.3% higher line coverage and 30.9% higher branch coverage than the agent-only baseline. A comparison with CLAUDE CODE confirms the framework dynamically adapts its policy to optimize each agent's unique strengths. SCATE also consistently outperforms state-of-the-art non-agentic approaches across all metrics.

Problem

• lazy generation
• automated test generation
• code coverage
• human-in-the-loop supervision
• coding agents

Innovation

• adaptive supervision
• coding agents
• test generation
• contextual bandit
• code coverage