Scaling LLM Reasoning from Minimal Labels: A Semi-Supervised Framework with a Lightweight Verifier

2026-06-15

AI Summary

This work addresses the high cost and limited scalability of supervised reasoning training for large language models, which typically relies on extensive human-annotated answers. The authors propose a semi-supervised framework that reframes reasoning correctness verification as a data generation mechanism. By employing a lightweight correctness classifier combined with entropy-based confidence filtering, the method selects high-quality reasoning trajectories from a minimal set of labeled examples and generates pseudo-labels for model fine-tuning. This approach replaces expensive answer-level supervision with efficient verification, substantially reducing annotation dependence. Experiments on Orca-Math and GQA demonstrate that the method achieves comparable performance using only one-tenth to one-fifteenth of the original labeled data, confirming its efficiency and scalability.

Abstract

For the development of Large language models (LLMs), recent approaches to generating pseudo intermediate reasoning have shown remarkable progress. But they typically rely on large numbers of correctly annotated answers to assess reasoning quality. This paper presents a semi-supervised framework that scales reasoning learning from minimal supervision, turning reasoning verification itself into a data creation mechanism. We train a lightweight reasoning-correctness classifier on only a few labeled samples, which judges whether intermediate reasoning traces generated by an LLM are valid. Furthermore, an entropy-based confidence threshold filters out unreliable samples, and the remaining high-confidence reasoning traces are used to fine-tune the model. Experiments on Verifiable Math Problems (Orca-Math subset) and Question Answering on Image Scene Graphs (GQA) with Visual Programming show that our method achieves accuracy comparable to using 10-15x more labeled data. Ablation analyses confirm that both the classifier and entropy filtering are essential for scalable and noise-resistant pseudo-labeling. By replacing expensive answer-level supervision with lightweight reasoning verification, our method provides a practical path toward constructing large-scale reasoning resources and paves the way for future autonomous reasoning systems that learn from minimal human input.

Problem

• semi-supervised learning
• reasoning verification
• minimal supervision
• pseudo-labeling
• large language models

Innovation

• semi-supervised learning
• reasoning verification
• lightweight verifier
• pseudo-labeling
• entropy-based filtering