A Benchmark and Framework for Evaluating Next Action Predictions in Spreadsheets

2026-06-11

AI Summary

This work addresses the lack of intelligent next-action prediction capabilities in spreadsheets—akin to code completion—and the absence of public datasets with rich editing histories, compounded by a complex action space involving spatiotemporal and composite operations. To bridge this gap, the study introduces the first standardized benchmark and an online dynamic evaluation framework for spreadsheet action prediction. Built upon 52 high-quality, human-reconstructed interaction sequences comprising 12K operations, the framework evaluates prediction performance in real time after each user action until task completion. The research establishes diverse baselines, including zero-shot large language models, fine-tuned small models, and classical methods, and constructs action sequences via heuristic generation refined by LLMs. Comprehensive experiments systematically analyze the impact of action characteristics, false positive rates, efficiency, and user profiles on predictive performance, laying a foundation for intelligent spreadsheet assistance.

Abstract

Predictive code completion greatly accelerates how quickly developers work. In spreadsheets, despite being much more common, such auto-completion features are virtually non-existent. To address this gap, we introduce a benchmark for systems that observe a sequence of user actions in a spreadsheet and predict future actions. Two challenges are (1) the absence of edit histories in public spreadsheet corpora and (2) the complex space of spreadsheet actions (spatial, temporal, composite). To address (1), we manually curate 52 sequences of 12K actions that recreate spreadsheets from public corpora, seeded by parametrized heuristics and LLM refinement. To address (2), we propose an online evaluation that expects a prediction after each user action, accepts or rejects that prediction, updates the future actions upon acceptance, and repeats this until the target spreadsheet is obtained. We use multiple baseline predictors (including zero-shot LLMs, fine-tuned SLMs, and classical models) and analyze different properties that our benchmark teaches us, including but not limited to: properties of saved actions and false positives, efficiency, effect of user profiles, effect of triggers, and effect of context.

Problem

• next action prediction
• spreadsheet
• code completion
• edit history
• predictive modeling

Innovation

• next action prediction
• spreadsheet interaction
• online evaluation framework
• action sequence benchmark
• LLM-based prediction