Optimal Stopping of Self-Refining Foundation Models

2026-08-11

AI Summary

This work addresses the optimal stopping problem in the self-refinement process of foundation models, aiming to determine the best termination point with minimal computational cost. The iterative refinement procedure is formulated as an optimal stopping problem that balances expected performance gains against computational overhead. The paper introduces, for the first time, an efficient and computable stopping strategy that dynamically decides when to halt refinement by integrating stochastic approximation, in-context learning, and external feedback mechanisms. Experimental results on code generation benchmarks demonstrate that the proposed method significantly reduces computational costs while maintaining or even surpassing the performance of existing approaches, thereby validating its effectiveness and practicality.

Abstract

Foundation models can improve their outputs through a self-refinement process driven by external feedback. In this process, the model is embedded in an iterative loop where it generates outputs, receives feedback from verifiers, and refines its responses through in-context learning. Following a novel approach, we formalize this process as an optimal stopping problem where the number of refinement iterations is decided based on expected improvement relative to cost. We derive optimal stopping policies and show that they can be efficiently computed through stochastic approximation. To evaluate our approach experimentally, we apply it to a coding benchmark for foundation models. The empirical results show that our stopping policies are significantly more cost-efficient than stopping policies proposed in prior work.

Problem

• optimal stopping
• self-refining
• foundation models
• cost-efficiency
• iterative refinement

Innovation

• optimal stopping
• self-refinement
• foundation models
• stochastic approximation
• cost-efficiency