Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning

2026-05-26

AI Summary

This work addresses the limitation of conventional code generation methods, which often produce redundant reasoning paths during test-time sampling due to independent execution, thereby failing to explore diverse problem-solving strategies and wasting computational resources. To overcome this, the authors propose Coordinated pass@K Policy Optimization (CPPO), a planner-solver collaborative framework in which a planner generates diverse high-level strategy tuples, and a shared solver executes them in parallel. Credit assignment is refined through a multiplicative reward mechanism, \( R_{\text{plan}} = J_\psi \cdot R_{\text{out}} \), which effectively encourages strategic diversity. Evaluated on the APPS, CodeContests, and LiveCodeBench-v6 benchmarks, CPPO significantly outperforms existing approaches, improving pass@4 from 0.588 to 0.748 on Qwen3.5-9B.

Abstract

Repeated sampling with a verifier is the standard way to allocate test-time compute for code generation, with pass@$K$ as the canonical metric. Yet the standard policy class draws $K$ independent samples from a single answer distribution, so attempts often collapse onto near-duplicate reasoning paths and waste the budget on redundant rollouts. This failure is costly in competitive programming, where many problems admit multiple distinct algorithmic strategies and pass@$K$ requires only one correct attempt. We propose Coordinated Pass@$K$ Policy Optimization (CPPO), which turns pass@$K$ generation into joint exploration over strategies: a planner emits a tuple of $K{=}4$ alternative high-level methods, and a shared solver attempts one solution per method. CPPO trains this joint policy with a multiplicative planner reward, $R_{\mathrm{plan}} = J_ψ\cdot R_{\mathrm{out}}$, assigning credit only to valid strategy tuples that lead to verifier-confirmed pass@$K$ success. Across APPS, CodeContests, and LiveCodeBench-v6, CPPO improves pass@$4$ over direct sampling, planning baselines, planner-only SFT, and pass@$K$-oriented RL under the same $K{=}4$ solver-attempt budget, with statistically significant gains on six of nine model--benchmark cells. The largest single gain is $+0.16$ on Qwen3.5-9B LiveCodeBench-v6 over the strongest baseline, PKPO ($0.588 \rightarrow 0.748$; paired bootstrap, $p < 0.05$).

Problem

• code generation
• pass@K
• reasoning diversity
• algorithmic strategies
• test-time compute allocation

Innovation

• Coordinated Pass@K
• policy optimization
• code reasoning
• strategy diversity
• planner-solver architecture