ESG-Bench: Benchmarking Long-Context ESG Reports for Hallucination Mitigation

2026-03-13

AI Summary

This study addresses the challenge of hallucinations in large language models (LLMs) when automatically analyzing lengthy and complex ESG reports, which undermines their reliability in compliance-sensitive applications. To this end, the authors introduce ESG-Bench, the first fine-grained hallucination annotation benchmark tailored to the ESG domain, framing report analysis as a verifiable question-answering task and incorporating human annotations to assess factual consistency in model outputs. The work proposes a novel approach combining Chain-of-Thought prompting with targeted fine-tuning, which significantly reduces hallucination rates compared to standard prompting or direct fine-tuning alone. This method not only achieves state-of-the-art performance on ESG-Bench but also demonstrates strong generalization capabilities on non-ESG question-answering benchmarks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

As corporate responsibility increasingly incorporates environmental, social, and governance (ESG) criteria, ESG reporting is becoming a legal requirement in many regions and a key channel for documenting sustainability practices and assessing firms' long-term and ethical performance. However, the length and complexity of ESG disclosures make them difficult to interpret and automate the analysis reliably. To support scalable and trustworthy analysis, this paper introduces ESG-Bench, a benchmark dataset for ESG report understanding and hallucination mitigation in large language models (LLMs). ESG-Bench contains human-annotated question-answer (QA) pairs grounded in real-world ESG report contexts, with fine-grained labels indicating whether model outputs are factually supported or hallucinated. Framing ESG report analysis as a QA task with verifiability constraints enables systematic evaluation of LLMs' ability to extract and reason over ESG content and provides a new use case: mitigating hallucinations in socially sensitive, compliance-critical settings. We design task-specific Chain-of-Thought (CoT) prompting strategies and fine-tune multiple state-of-the-art LLMs on ESG-Bench using CoT-annotated rationales. Our experiments show that these CoT-based methods substantially outperform standard prompting and direct fine-tuning in reducing hallucinations, and that the gains transfer to existing QA benchmarks beyond the ESG domain.

Problem

• ESG reporting
• hallucination
• long-context understanding
• large language models
• factuality

Innovation

• ESG-Bench
• hallucination mitigation
• long-context understanding
• Chain-of-Thought prompting
• fact-grounded QA