VeriFin: A Neurosymbolic Framework for Verifying LLM-Generated Financial Claims

2026-08-10

AI Summary

This work addresses the challenge of factually incorrect yet semantically plausible numerical statements generated by large language models in financial contexts, which often stem from misaligned reporting periods, units, or formulas and evade detection by conventional validation methods. To tackle this, the authors propose VeriFin, the first auditable verification framework that integrates neural and symbolic approaches: it anchors numerical claims to XBRL filing facts, derives computational formulas from textual descriptions, and employs the Z3 solver to verify logical consistency. Evaluated on XBRLFiling and FinanceBench, VeriFin achieves zero false acceptance rates, substantially outperforming baseline methods. Furthermore, its conflict localization mechanism—based on unsatisfiable cores—enables precise error diagnosis and correction, attaining a maximum recovery rate of 69.9%.

Abstract

Large language models often produce plausible numerical claims from financial filings while using the wrong reporting period, unit, line item, or formula. Verifying such claims requires more than rechecking arithmetic: a verifier must ground the relevant facts, establish the authorized calculation, and determine whether the candidate value follows from both. We propose VeriFin, a neurosymbolic verification framework for numerical financial question answering that grounds operands in filed XBRL facts, derives calculations from the question, filing linkbases, or documented metric definitions, and checks claims using Z3. When a claim is inconsistent, solver-derived unsatisfiable cores identify the conflicting facts, formula, and candidate value, enabling targeted repair. We evaluate VeriFin on XBRLFiling, a new 600-question benchmark constructed from 10-K filings of 28 U.S. companies, and on FinanceBench. On fixed candidate pools shared by all verification methods, VeriFin accepts none of the incorrect claims, whereas the baselines accept 6 to 92 incorrect claims on XBRLFiling's 600 claims and 4 to 21 incorrect claims on FinanceBench's 67 claims. The zero false-accept result persists across multiple answer-generator models. Solver-derived feedback further improves repair, achieving up to 69.9% recovery among true catches. These results show that source-grounded symbolic verification can provide a reliable and auditable acceptance control for LLM-generated financial claims.

Problem

• financial claim verification
• large language models
• numerical reasoning
• XBRL
• fact grounding

Innovation

• neurosymbolic verification
• financial claim validation
• XBRL grounding
• Z3 solver
• unsatisfiable core