TIPCODER: Reinforcement Learning Boosted Test-time Instruction Proposer for Code Generation

2026-09-02

AI Summary

为解决代码生成中因约束缺失、边缘情况忽视或误导性推理路径导致的问题，提出TipCoder，通过强化学习优化指令提议，并在测试时生成基于提示的解决方案。

Abstract

Test-time scaling for code generation typically explores the solution space by sampling multiple programs from a fixed instruction. We study a complementary direction: instance-level instruction-space exploration. Our observation is that many coding failures stem from missing constraints, overlooked edge cases, or misleading reasoning paths induced by the original prompt. To address this, we propose TipCoder, a test-time instruction proposer that generates problem-specific auxiliary tips before code synthesis. TipCoder distills multi-turn debugging trajectories into proactive guidance and further optimizes the Proposer with reinforcement learning using a marginal-utility reward. At inference time, it generates both a base solution and a tip-guided solution, and applies a Reward Model for post-hoc selection. This exploration-selection design allows tips to expose additional candidate potential while reducing regressions from unnecessary guidance. Across the evaluated code-generation benchmarks and target Code LLMs, TipCoder provides a consistent instruction-level test-time scaling strategy, comparing favorably with stochastic sampling and generic prompt optimization baselines under a shared reward-model-based selection protocol.

Problem

• code generation
• instruction-space exploration
• coding failures

Innovation

• test-time instruction proposer
• reinforcement learning
• marginal-utility reward
• code generation
• reward model