ICBCBench: An Industry Consortium Benchmark for Financial Deep Research

2026-06-15

AI Summary

This study addresses the absence of a unified evaluation framework in financial deep research that jointly assesses both retrieval-reasoning accuracy and end-to-end report quality. To bridge this gap, the authors propose the first dual-track benchmark, collaboratively developed by finance and academic experts, which integrates objective question answering—designed to verify factual accuracy—with subjective long-form report evaluation encompassing expert alignment, citation consistency, and source credibility. This holistic approach enables coordinated assessment of an agent’s retrieval, reasoning, and generation capabilities. Experimental results reveal that prevailing large language models exhibit significant deficiencies in complex reasoning, evidentiary support for claims, and overall report quality, thereby exposing a critical performance gap between current technical capabilities and real-world industry demands.

Abstract

With the rapid advancement of Deep Research Agents in knowledge-intensive domains such as finance, establishing reliable and domain-aligned evaluation standards remains a critical challenge. Existing benchmarks focus on either closed-ended question answering or open-ended report evaluation, failing to jointly capture retrieval-reasoning accuracy and end-to-end research quality required in real-world workflows. We introduce ICBCBench, a consortium-driven benchmark for financial deep research, developed in collaboration with domain experts from a broad range of financial institutions and academia, involving over 50 experts across more than 40 organizations. It adopts a dual-track paradigm integrating objective tasks with verifiable answers and subjective long-form report evaluation, enabling complementary assessment of retrieval-reasoning accuracy and end-to-end report quality in terms of expert alignment, citation consistency, and source quality. Experiments on state-of-the-art DRAs and large language models reveal substantial gaps in complex reasoning, factual grounding, and report quality, highlighting the challenges of achieving industry-level performance. Our dataset and evaluation framework are available at https://github.com/DeepFin-Intelligence/ICBCBench.

Problem

• Deep Research Agents
• financial benchmark
• evaluation standards
• retrieval-reasoning accuracy
• end-to-end research quality

Innovation

• Deep Research Agents
• Financial Benchmarking
• Dual-Track Evaluation
• Retrieval-Reasoning Accuracy
• End-to-End Report Quality