ExecuCritic: Calibrated Critic Shaping for Code Generation with Verifiable Rewards

2026-09-14

AI Summary

提出ExecuCritic框架，通过训练编码器和批评者来解决代码生成中的奖励分配问题，提高代码生成的准确性。

Abstract

Execution feedback is a useful supervision signal for code models because unit tests are objective and directly measure program correctness. Its weakness is that an entire program is often reduced to one pass or fail bit, leaving RLVR to solve a difficult credit assignment problem. At the same time, coding systems often include separate reviewer or tester roles, but these critics are usually prompted rather than trained and are not calibrated against execution. We propose ExecuCritic, a joint training framework in which a coder and a critic are updated on the same execution rollouts. The critic predicts pass or fail outcomes and gives short diagnostic feedback; the coder uses this signal only when the critic agrees with the executor on the current rollout group. Across eight code benchmarks and two recent open backbones, ExecuCritic improves over GRPO without a critic, prompted reviewer systems and scalar reward model baselines, while requiring fewer policy gradient steps and fewer sandbox executions. Ablations and reliability analyses suggest that the gains come from better credit assignment rather than larger sampling budgets.

Problem

• execution feedback
• credit assignment problem
• critic calibration

Innovation

• ExecuCritic
• joint training framework
• credit assignment
• execution feedback
• verifiable rewards