When the Chain Breaks: Interactive Diagnosis of LLM Chain-of-Thought Reasoning Errors

2026-03-22

AI Summary

This work addresses the challenge of diagnosing errors in chain-of-thought (CoT) reasoning generated by large language models, which are often verbose and prone to logical or factual inaccuracies. To this end, the authors propose the first step-level error detection method that integrates external fact-checking with symbolic logical verification. They further develop ReasonDiag, an interactive visualization system that combines arc diagrams and hierarchical node-link graphs to reveal the reasoning flow and trace error propagation paths. Through technical evaluation, two case studies, and user interviews with 16 participants, the study demonstrates that ReasonDiag effectively supports users in comprehending complex reasoning processes, accurately identifying erroneous steps, and tracing underlying root causes.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Current Large Language Models (LLMs), especially Large Reasoning Models, can generate Chain-of-Thought (CoT) reasoning traces to illustrate how they produce final outputs, thereby facilitating trust calibration for users. However, these CoT reasoning traces are usually lengthy and tedious, and can contain various issues, such as logical and factual errors, which make it difficult for users to interpret the reasoning traces efficiently and accurately. To address these challenges, we develop an error detection pipeline that combines external fact-checking with symbolic formal logical validation to identify errors at the step level. Building on this pipeline, we propose ReasonDiag, an interactive visualization system for diagnosing CoT reasoning traces. ReasonDiag provides 1) an integrated arc diagram to show reasoning-step distributions and error-propagation patterns, and 2) a hierarchical node-link diagram to visualize high-level reasoning flows and premise dependencies. We evaluate ReasonDiag through a technical evaluation for the error detection pipeline, two case studies, and user interviews with 16 participants. The results indicate that ReasonDiag helps users effectively understand CoT reasoning traces, identify erroneous steps, and determine their root causes.

Problem

• Chain-of-Thought
• reasoning errors
• error diagnosis
• Large Language Models
• interpretability

Innovation

• Chain-of-Thought
• Error Detection
• Interactive Visualization
• Logical Validation
• Fact-Checking