STRIVE: Structured Reasoning for Self-Improvement in Claim Verification

2025-02-17

AI Summary

In claim verification, self-improvement methods suffer from erroneous reasoning contamination and performance degradation due to low-quality reasoning chains misaligned with binary truth labels. To address this, we propose a structured reasoning framework specifically designed for claim verification, comprising three stages: claim decomposition, entity analysis, and evidence-anchored verification. We introduce the first multi-granularity supervision signal design, incorporating structural correctness constraints into the self-improvement paradigm. Our approach integrates structured prompt engineering, phased fine-tuning (warm-up followed by self-improvement), and a dual-filtering mechanism based on both reasoning-chain quality and answer consistency. Evaluated on the HOVER dataset, our method achieves a 31.4% improvement over strong baselines and outperforms standard chain-of-thought approaches by 20.7%, significantly enhancing reasoning reliability and verification accuracy.

Technology Category

• Knowledge Representation and Reasoning: Argumentation
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Claim verification is the task of determining whether a claim is supported or refuted by evidence. Self-improvement methods, where reasoning chains are generated and those leading to correct results are selected for training, have succeeded in tasks like mathematical problem solving. However, in claim verification, this approach struggles. Low-quality reasoning chains may falsely match binary truth labels, introducing faulty reasoning into the self-improvement process and ultimately degrading performance. To address this, we propose STRIVE: Structured Reasoning for Self-Improved Verification. Our method introduces a structured reasoning design with Claim Decomposition, Entity Analysis, and Evidence Grounding Verification. These components improve reasoning quality, reduce errors, and provide additional supervision signals for self-improvement. STRIVE begins with a warm-up phase, where the base model is fine-tuned on a small number of annotated examples to learn the structured reasoning design. It is then applied to generate reasoning chains for all training examples, selecting only those that are correct and structurally sound for subsequent self-improvement training. We demonstrate that STRIVE achieves significant improvements over baseline models, with a 31.4% performance gain over the base model and 20.7% over Chain of Thought on the HOVER datasets, highlighting its effectiveness.

Problem

• Enhances claim verification accuracy
• Reduces faulty reasoning chains
• Improves self-improvement methods performance

Innovation

• Structured reasoning design
• Claim decomposition technique
• Evidence grounding verification