Thucy: An LLM-based Multi-Agent System for Claim Verification across Relational Databases

2025-12-02

AI Summary

Addressing the challenge of automated fact verification for claims drawn from multi-source, heterogeneous relational databases—spanning domains such as politics, economics, and healthcare—this paper proposes the first end-to-end verification framework that operates without prior knowledge of database schemas. Our approach leverages large language models to construct a multi-agent system that autonomously performs database discovery, schema understanding, cross-database relational reasoning, and interpretable natural-language-to-SQL generation. Each verification decision is accompanied by a supporting SQL query serving as auditable evidence. Unlike prior methods restricted to single-table, small-scale data, our framework enables genuine cross-database and cross-table claim verification. Evaluated on the TabFact benchmark, it achieves 94.3% accuracy—surpassing the state-of-the-art by 5.6 percentage points—and significantly advances structured-data-driven fact-checking in real-world settings, both in efficacy and interpretability.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Reasoning with Beliefs
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

In today's age, it is becoming increasingly difficult to decipher truth from lies. Every day, politicians, media outlets, and public figures make conflicting claims$unicode{x2014}$often about topics that can, in principle, be verified against structured data. For instance, statements about crime rates, economic growth or healthcare can all be verified against official public records and structured datasets. Building a system that can automatically do that would have sounded like science fiction just a few years ago. Yet, with the extraordinary progress in LLMs and agentic AI, this is now within reach. Still, there remains a striking gap between what is technically possible and what is being demonstrated by recent work. Most existing verification systems operate only on small, single-table databases$unicode{x2014}$typically a few hundred rows$unicode{x2014}$that conveniently fit within an LLM's context window. In this paper we report our progress on Thucy, the first cross-database, cross-table multi-agent claim verification system that also provides concrete evidence for each verification verdict. Thucy remains completely agnostic to the underlying data sources before deployment and must therefore autonomously discover, inspect, and reason over all available relational databases to verify claims. Importantly, Thucy also reports the exact SQL queries that support its verdict (whether the claim is accurate or not) offering full transparency to expert users familiar with SQL. When evaluated on the TabFact dataset$unicode{x2014}$the standard benchmark for fact verification over structured data$unicode{x2014}$Thucy surpasses the previous state of the art by 5.6 percentage points in accuracy (94.3% vs. 88.7%).

Problem

• Verifies claims across multiple relational databases autonomously.
• Provides SQL evidence for verification transparency and trust.
• Improves accuracy over existing single-table verification systems.

Innovation

• Multi-agent system verifies claims across relational databases autonomously
• Generates SQL queries as transparent evidence for verification verdicts
• Achieves state-of-the-art accuracy on TabFact benchmark dataset