Teach-to-Reason: Competition-Guided Reasoning with a Self-Improving Teacher

2026-06-24

AI Summary

This work addresses the challenge that existing visual question answering (VQA) models for chest X-rays rely solely on answer-level reward signals, which often degrade and hinder effective optimization of medical reasoning chains. To overcome this limitation, the authors propose the Teach-to-Reason framework, which jointly trains a self-evolving teacher model and a competition-guided reasoner. The framework introduces a comparative supervision mechanism to refine the reasoning process and incorporates instance-level dynamic rewards to compensate for insufficient original reward signals. By integrating reinforcement learning, chain-of-thought reasoning, and a self-competition strategy, the method significantly outperforms strong baselines across multiple open-ended chest X-ray VQA benchmarks, demonstrating markedly improved medical reasoning quality.

Abstract

Chest X-ray visual question answering (CXR VQA) requires models not only to predict correct answers, but also to produce reliable medical reasoning. However, existing reinforcement-learning-based training typically relies on answer-level rewards, which are often too coarse to improve chain-of-thought (CoT) quality and can become ineffective when group-level advantages collapse to zero. We propose \textbf{Teach-to-Reason (T2R)}, a framework that introduces comparison-based supervision into CoT optimization through a self-improving \emph{Teacher} and a competition-guided \emph{Reasoner}. As the Teacher is iteratively strengthened via self-competition, the Reasoner is optimized against progressively stronger Teacher-generated references. We further introduce a case-wise reward design that preserves the original reward-induced positive/negative partition when it is informative, and restores supervision from competition scores when the original reward signal degenerates. Experiments on multiple CXR open-ended VQA benchmarks show that T2R consistently outperforms strong baselines, indicating that comparison-based supervision, when integrated in a controlled and principled manner, provides a more effective training signal for reasoning optimization.

Problem

• Chest X-ray VQA
• chain-of-thought reasoning
• reinforcement learning
• reward sparsity
• medical reasoning

Innovation

• Teach-to-Reason
• comparison-based supervision
• self-improving teacher
• competition-guided reasoning
• chain-of-thought optimization