Claim-Level Reliability Assessment for Efficient Test-Time Reasoning

2026-08-12

AI Summary

This work addresses the challenge that large language models often dilute critical errors during reasoning due to holistic trajectory evaluation, making it difficult to detect high-confidence yet incorrect reasoning paths. The authors propose a training-agnostic Claim-Level Reliability (CLR) framework that reallocates computational resources at test time from redundant sampling to targeted verification. CLR extracts decision-critical claims and leverages the asymmetry between constructive solutions and counterarguments to perform semantic falsification, efficiently eliminating erroneous trajectories through single-point fatal flaws. The framework integrates claim extraction, semantic falsification, and a nonlinear scoring mechanism. Evaluated across four mainstream models and reasoning benchmarks, CLR substantially outperforms both pass@1 and self-consistency baselines—e.g., achieving a 27.15 percentage point accuracy gain on GPT-OSS-20B/CMIMC25 while reducing token consumption by 37.0%.

Abstract

We propose claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR), a training-free framework that reallocates test-time compute from additional solution sampling to targeted verification. Since whole-trace evaluation often obscures decisive errors due to signal dilution from routine tokens, CLR condenses each reasoning trace into a compact set of decision-critical claims, thereby isolating its logical anchors. Furthermore, recognizing the inherent difficulty of generating entirely correct solutions under fixed model capabilities, CLR shifts the focus to semantic falsification. This approach exploits a fundamental asymmetry between solution construction and claim refutation. Constructing a valid solution requires a flawless reasoning path, whereas refuting an incorrect claim requires identifying only a single decisive flaw. This targeted search for negative evidence systematically compresses the survival space of high-confidence incorrect traces, effectively suppressing erroneous consensus via nonlinear reliability scoring. Across four LLMs and four reasoning benchmarks under matched budgets, CLR generally improves upon pass@1 and self-consistency. On GPT-OSS-20B/CMIMC25, for instance, CLR exceeds pass@1 by 27.15 percentage-points and raises self-consistency accuracy from 77.50\% to 82.19\% with 37.0\% fewer tokens.

Problem

• claim-level reliability
• test-time reasoning
• semantic falsification
• reasoning trace evaluation
• error suppression

Innovation

• claim-level falsification
• test-time scaling
• reasoning verification
• semantic falsification
• reliability scoring