Problem
Research questions and friction points this paper is trying to address.
causal reasoning
process distillation
reasoning errors
step-dependent chains
right-label-wrong-reasoning
CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning
🤖 AI Summary
This work addresses the limitation of standard process distillation in tasks requiring causal-chain reasoning—such as clinical diagnosis and legal judgment—where student models often commit early errors in their reasoning trajectories that propagate undetected, leading to correct conclusions derived through flawed logic. To mitigate this, the authors propose CausalOPD, a novel framework featuring “first-error-step supervision.” It leverages a knowledge-augmented teacher to generate reasoning trajectories aligned with domain-specific causal constraints, precisely identifies the first step in the student’s trajectory that violates these constraints, and applies short-horizon reinforcement to repair the trajectory based on the verified prefix. Additionally, a curriculum learning strategy is introduced that progressively advances through causal reasoning stages. Experiments demonstrate that CausalOPD improves average path accuracy by 23.4 percentage points across three domains, reduces the rate of “correct conclusion but incorrect reasoning” from 15.7% to 4.4%, and enables an 8B-parameter student model to surpass all compared closed-source models in reasoning path correctness.
📝 Abstract
Many critical reasoning tasks, including clinical diagnosis, legal judgment, and industrial fault diagnosis, require step-dependent causal chains in which early errors propagate and correct conclusions can mask invalid reasoning. Although large language models perform well on such tasks, privacy, latency, and controllability motivate distillation into locally deployable models. Standard trajectory imitation does not correct process errors on the student's own rollout distribution. We propose CausalOPD, a curriculum online process distillation framework. A knowledge-augmented teacher first provides trajectories grounded in domain-specific causal rules, entity relations, and structural constraints. The student then generates on-policy trajectories, and the teacher identifies the first wrong step, defined as the earliest transition that verifiably violates available constraints. Starting from the verified prefix, short-horizon reinforcement learning repairs this localized failure. A causal-stage curriculum advances from evidence-level to mechanism-level and conclusion-level errors, following their propagation order. Across three domains, CausalOPD improves average path correctness by 23.4 percentage points over sequence-level online process distillation and reduces the right-label-wrong-reasoning rate from 15.7% to 4.4%. The domain-specific 8B students also surpass both evaluated proprietary references in path correctness across all domains.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Causal Reasoning
Process Distillation
First-Wrong-Step Supervision
Curriculum Learning
On-Policy Correction
🔎 Similar Papers
No similar papers found.
