FISCAL: Financial Synthetic Claim-document Augmented Learning for Efficient Fact-Checking

2025-11-24

AI Summary

To address the severe hallucination, high computational overhead, and reliance on ultra-large parameter counts that plague large language models (LLMs) in financial fact-checking, this paper proposes FISCAL: a novel framework featuring a domain-specific, modular synthetic data generation method tailored for finance. It constructs high-quality claim-document pairs for training. Leveraging this data, we train MiniCheck-FISCAL—a lightweight, parameter-efficient verification model. Despite its small scale, MiniCheck-FISCAL surpasses Gemini-1.5 Flash and achieves performance competitive with Mixtral-8x22B (20× larger) and GPT-4o/Claude-3.5 on FinDVer and Fin-Fact benchmarks. The core innovation lies in explicitly embedding financial domain knowledge into both synthetic data generation and augmentation processes, enabling high accuracy, efficiency, and practicality for numerical financial claim verification—without requiring massive models.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Financial applications of large language models (LLMs) require factual reliability and computational efficiency, yet current systems often hallucinate details and depend on prohibitively large models. We propose FISCAL (Financial Synthetic Claim-Document Augmented Learning), a modular framework for generating synthetic data tailored to financial fact-checking. Using FISCAL, we generate a dataset called FISCAL-data and use it to train MiniCheck-FISCAL, a lightweight verifier for numerical financial claims. MiniCheck-FISCAL outperforms its baseline, surpasses GPT-3.5 Turbo and other open-source peers of similar size, and approaches the accuracy of much larger systems (20x), such as Mixtral-8x22B and Command R+. On external datasets FinDVer and Fin-Fact, it rivals GPT-4o and Claude-3.5 while outperforming Gemini-1.5 Flash. These results show that domain-specific synthetic data, combined with efficient fine-tuning, enables compact models to achieve state-of-the-art accuracy, robustness, and scalability for practical financial AI. The dataset and scripts are available in the project repository (link provided in the paper).

Problem

• Addressing factual unreliability in financial LLMs
• Reducing computational costs of large financial models
• Generating domain-specific synthetic data for fact-checking

Innovation

• Generates synthetic financial data for fact-checking
• Trains lightweight verifier models for numerical claims
• Uses domain-specific data for efficient fine-tuning