Back to the Future: A workbook time machine for spread sheet creation benchmarks

2026-08-07

AI Summary

Existing language models lack a systematic evaluation benchmark for generating spreadsheet multi-artifacts—such as formulas, charts, pivot tables, and conditional formatting. This work proposes the "Workbook Time Machine," which, for the first time, automatically constructs a large-scale corpus (wtmcorpus) by reverse-engineering publicly available workbooks to extract input–output–query triplets. Building upon this corpus, the authors introduce wtmbench, a comprehensive benchmark comprising 150 tasks spanning four artifact types and multiple levels of instruction complexity. The framework enables automated task generation and annotation, revealing critical insights into how query specificity, agent collaboration mechanisms, and API design influence model performance. This study establishes the first holistic evaluation framework for intelligent spreadsheet agents.

Abstract

We introduce the workbook time machine, a pipeline that automatically creates benchmarks evaluating the ability of language models to create derived objects in spreadsheets (formulas, charts, pivot tables, and conditional formatting). Applied to public workbook corpora, it produces wtmcorpus--a collection of (input workbook, output workbook, query) triples spanning four artifact types and varying complexity. From this corpus we curate wtmbench, a 150-task evaluation benchmark with queries at three levels of specificity. We evaluate existing spreadsheet manipulation agents and baselines on wtmbench across artifact types, step complexity, and instruction granularity. Our evaluations show that query specificity, agent orchestration, and interface API used to control spreadsheets play a big role in LLM performance on Excel tasks.

Problem

• spreadsheet
• benchmark
• language models
• derived objects
• evaluation

Innovation

• workbook time machine
• spreadsheet benchmark
• LLM evaluation
• derived spreadsheet artifacts
• query specificity