UniDataBench: Evaluating Data Analytics Agents Across Structured and Unstructured Data

2025-11-03

AI Summary

In real-world business settings, data reside across heterogeneous sources—including relational databases, NoSQL systems, and CSV/Excel files—yet no unified benchmark exists to evaluate intelligent agents’ capability for cross-source analytical reasoning. Method: We introduce UniDataBench, the first benchmark tailored to realistic business scenarios, supporting both structured and unstructured multi-source data and incorporating privacy-preserving data anonymization. We further propose ReActInsight, a large language model–based autonomous analytical agent that integrates ReAct-style reasoning, goal decomposition, cross-source relationship discovery, and self-correcting code generation. Contribution/Results: Experiments demonstrate that our framework significantly improves cross-source data analysis efficiency, achieving strong generalization and high accuracy on complex analytical tasks. UniDataBench establishes a novel evaluation paradigm for data analysis agents, while ReActInsight advances the state of autonomous, multi-source analytical reasoning—enabling more robust, scalable, and trustworthy business intelligence systems.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Multiagent Systems: Modeling other Agents
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Agentic search

Abstract

In the real business world, data is stored in a variety of sources, including structured relational databases, unstructured databases (e.g., NoSQL databases), or even CSV/excel files. The ability to extract reasonable insights across these diverse source is vital for business success. Existing benchmarks, however, are limited in assessing agents' capabilities across these diverse data types. To address this gap, we introduce UniDataBench, a comprehensive benchmark designed to evaluate the performance of data analytics agents in handling diverse data sources. Specifically, UniDataBench is originating from real-life industry analysis report and we then propose a pipeline to remove the privacy and sensitive information. It encompasses a wide array of datasets, including relational databases, CSV files to NoSQL data, reflecting real-world business scenarios, and provides unified framework to assess how effectively agents can explore multiple data formats, extract valuable insights, and generate meaningful summaries and recommendations. Based on UniDataBench, we propose a novel LLM-based agent named ReActInsight, an autonomous agent that performs end-to-end analysis over diverse data sources by automatically discovering cross-source linkages, decomposing goals, and generating robust, self-correcting code to extract actionable insights. Our benchmark and agent together provide a powerful framework for advancing the capabilities of data analytics agents in real-world applications.

Problem

• Evaluating analytics agents across diverse data sources
• Assessing capabilities on structured and unstructured data
• Providing unified framework for real-world business scenarios

Innovation

• Proposes UniDataBench benchmark for multi-format data evaluation
• Introduces ReActInsight agent with automated cross-source linkage discovery
• Generates self-correcting code for actionable insight extraction