SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

2026-06-29

AI Summary

This work addresses the limitations of existing spreadsheet benchmarks, which focus on isolated operations and fail to evaluate end-to-end, multi-step workflows across worksheets as encountered in real-world business settings. To bridge this gap, the study introduces the first evaluation framework tailored to complex commercial spreadsheet workflows, constructing a realistic benchmark based on authentic data that encompasses generation, debugging, and visualization tasks—each involving an average of 11.8 worksheets and 593.5 cell modifications. The authors further develop an expert-annotated taxonomy for task decomposition and failure attribution. Evaluating state-of-the-art large language models and existing LLM-powered spreadsheet tools within a unified multi-turn agent architecture reveals that even the best-performing model achieves only 34.89% overall accuracy, with debugging tasks scoring as low as 12.00%, highlighting critical bottlenecks in target cell localization and structural understanding of spreadsheets.

Abstract

Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making. However, most existing spreadsheet benchmarks evaluate isolated operations such as single-formula generation or local cell edits, and therefore fail to capture end-to-end workflows in realistic business settings. We introduce \textsc{SpreadsheetBench 2}, a workflow-level benchmark for spreadsheet agents that covers three task categories: generation, debugging, and visualization. The benchmark is constructed from authentic business data, including financial reports and corporate filings, and is annotated and validated by domain experts. The benchmark contains 321 tasks; each instance averages 11.8 worksheets and requires 593.5 cell modifications, reflecting large multi-sheet workbooks with cross-sheet dependencies. We evaluate eight frontier large language models under a unified multi-turn agent scaffold, and additionally include several LLM-based spreadsheet products as complementary baselines. Results show that current systems remain far from reliable on real-world workflows: the best model achieves 34.89\% overall task accuracy, and debugging accuracy is as low as 12.00\%. Trajectory analysis and a failure taxonomy further indicate that insufficient spreadsheet inspection and incorrect target-cell selection are the dominant bottlenecks. Together, these findings position \textsc{SpreadsheetBench 2} as a challenging testbed for advancing reliable spreadsheet automation. Project page: https://spreadsheetbench.github.io/

Problem

• spreadsheet
• benchmark
• workflow
• business analysis
• automation

Innovation

• spreadsheet agents
• end-to-end workflows
• benchmark
• multi-sheet dependencies
• LLM evaluation