Know Your Limits : On the Faithfulness of LLMs as Solvers and Autoformalizers in Legal Reasoning

2026-06-14

AI Summary

This study investigates whether large language models (LLMs) perform faithful logical reasoning in legal tasks or merely rely on heuristic approximations. By systematically evaluating three paradigms—pure LLM classification, LLM-based formalization followed by natural language inference, and rigorous formal reasoning using the Z3 solver—on a manually relabeled subset of ContractNLI, the work reveals a systematic gap between legal pragmatic interpretation and formal entailment. It identifies three characteristic failure modes in LLM-driven formalization, including “scope sanitization.” Although formalization improves task accuracy, the experiments demonstrate that all models exhibit pervasive logical unfaithfulness, underscoring that high predictive accuracy does not guarantee logical correctness.

Abstract

Large Language Models (LLMs) achieve strong performance on reasoning tasks, but whether this reflects faithful logical inference or heuristic approximation remains unclear. We study this question in legal entailment by comparing three paradigms, including pure LLM classification, LLM-based Formal Reasoning, and solver-based Formal Reasoning using the Z3 SMT solver, on a re-annotated subset of ContractNLI across five LLMs. Our re-annotation reveals a systematic and measurable gap between pragmatic legal interpretation and strict formal entailment, where a substantial proportion of legally sound inferences are not formally grounded without additional unstated assumptions. While introducing formal structure improves accuracy, with LLM-based Formal Reasoning achieving the highest benchmark performance, we show that this gain does not imply faithful reasoning. We identify three recurring failure modes: scope laundering, where LLMs report solver-inconsistent classifications without executing the underlying formal reasoning, producing conclusions that appear logically grounded but are not; implicit constraint blindness, where LLMs overlook logical constraints present in formal representations; and program synthesis failures, where LLMs generate incorrect Z3 code despite structured prompting. Critically, scope laundering persists across all models, raising serious concerns about the faithfulness of LLM-based formal reasoning as a proxy for symbolic execution. These results reveal a fundamental gap between benchmark accuracy and logical faithfulness.

Problem

• faithfulness
• legal reasoning
• formal entailment
• large language models
• logical inference

Innovation

• faithful reasoning
• formal entailment
• legal NLI
• SMT solving
• LLM failure modes