Self-Taught Self-Correction for Small Language Models

2025-03-11

AI Summary

Small language models (SLMs) lack intrinsic self-correction capabilities, limiting their reliability in reasoning tasks without external tools or large-model assistance. Method: We propose Self-Taught Self-Correction (STaSC), a purely self-supervised, iterative fine-tuning paradigm that enables SLMs to autonomously generate high-quality correction samples. STaSC integrates confidence-driven sample filtering, iterative instruction fine-tuning, self-distillation, and lightweight parameter-efficient adaptation—requiring no external supervision or auxiliary models. Contribution/Results: Evaluated on standard question-answering benchmarks, STaSC significantly improves accuracy, providing the first empirical evidence that SLMs possess trainable, end-to-end self-correcting mechanisms. All code and optimized lightweight models are publicly released, establishing a new pathway for robust, resource-efficient inference in constrained environments.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Although large language models (LLMs) have achieved remarkable performance across various tasks, they remain prone to errors. A key challenge is enabling them to self-correct. While prior research has relied on external tools or large proprietary models, this work explores self-correction in small language models (SLMs) through iterative fine-tuning using solely self-generated data. We introduce the Self-Taught Self-Correction (STaSC) algorithm, which incorporates multiple algorithmic design choices. Experimental results on a question-answering task demonstrate that STaSC effectively learns self-correction, leading to significant performance improvements. Our analysis further provides insights into the mechanisms of self-correction and the impact of different design choices on learning dynamics and overall performance. To support future research, we release our user-friendly codebase and lightweight models.

Problem

• Enabling small language models to self-correct errors
• Exploring self-correction without external tools or large models
• Improving performance through iterative fine-tuning with self-generated data

Innovation

• Self-Taught Self-Correction algorithm for SLMs
• Iterative fine-tuning with self-generated data
• User-friendly codebase and lightweight models