Self-Improvement as Coherence Optimization: A Theoretical Account

2026-01-20

AI Summary

Existing unsupervised self-improvement mechanisms for language models—such as debate, bootstrapping, and internal consistency maximization—lack a unified theoretical foundation. This work formalizes such self-improvement as a consistency optimization problem: finding the most compressible and jointly predictable mapping from context to behavior. We prove that this formulation is equivalent to description-length regularization and achieves theoretical optimality in semi-supervised learning. Grounded in information theory and leveraging pretraining-guided regularization, our framework not only provides a principled theoretical basis for existing methods but also predicts the conditions under which they succeed or fail. Preliminary experiments validate the effectiveness of the proposed approach.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Can language models improve their accuracy without external supervision? Methods such as debate, bootstrap, and internal coherence maximization achieve this surprising feat, even matching golden finetuning performance. Yet why they work remains theoretically unclear. We show that they are all special cases of coherence optimization: finding a context-to-behavior mapping that's most compressible and jointly predictable. We prove that coherence optimization is equivalent to description-length regularization, and that among all such regularization schemes, it is optimal for semi-supervised learning when the regularizer is derived from a pretrained model. Our theory, supported by preliminary experiments, explains why feedback-free self-improvement works and predicts when it should succeed or fail.

Problem

• self-improvement
• coherence optimization
• language models
• semi-supervised learning
• description-length regularization

Innovation

• coherence optimization
• description-length regularization
• self-improvement
• semi-supervised learning
• pretrained language models