Compartmentalised Agentic Reasoning for Clinical NLI

2025-09-12

AI Summary

Clinical natural language inference (NLI) suffers from loose reasoning structures, poor generalizability, and non-auditable inference processes. Method: This paper proposes CARENLI, a modular agent-based reasoning framework that decomposes clinical NLI into four distinct reasoning families, decoupling knowledge acquisition from logical inference. It introduces a dynamic routing mechanism to allocate reasoning tasks and a three-stage controllable pipeline—planning, verification, and correction—to explicitly model reasoning chains. Contribution/Results: CARENLI uncovers a cognitive gap in large language models (LLMs), revealing their reliance on heuristic strategies under reasoning ambiguity. Experiments across four mainstream LLMs demonstrate up to a 42-percentage-point improvement in reasoning fidelity, 98.0% accuracy in causal attribution, 81.2% success in risk-state abstraction, near-100% violation detection by the verifier, and significant reduction in cognitive errors by the corrector.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

A common assumption holds that scaling data and parameters yields increasingly structured, generalisable internal representations. We interrogate this assumption in clinical natural language inference (NLI) by adopting a benchmark decomposed into four reasoning families, Causal Attribution, Compositional Grounding, Epistemic Verification, and Risk State Abstraction, and introducing CARENLI, a Compartmentalised Agentic Reasoning for Clinical NLI that separates knowledge access from principled inference. CARENLI routes each premise, statement pair to a family specific solver and enforces auditable procedures via a planner, verifier, and refiner. Across four LLMs, CARENLI improves fidelity by up to 42 points, reaching 98.0% in Causal Attribution and 81.2% in Risk State Abstraction. Verifiers flag violations with near-ceiling reliability, while refiners correct a substantial share of epistemic errors. Remaining failures cluster in routing, identifying family classification as the main bottleneck. These results show that LLMs often retain relevant facts but default to heuristics when inference is underspecified, a dissociation CARENLI makes explicit while offering a framework for safer, auditable reasoning.

Problem

• Improves clinical NLI reasoning fidelity
• Separates knowledge access from inference
• Addresses heuristic defaults in LLMs

Innovation

• Compartmentalised agentic reasoning for clinical NLI
• Family-specific solvers with routing mechanism
• Planner-verifier-refiner framework for auditable procedures