microsoft office

To address the high cost and scalability limitations of expert-authored Excel tutorials, this paper proposes the first end-to-end automated framework that generates functionally complete, executable Excel tutorials directly from natural language task descriptions. Methodologically, it employs a large language model (LLM)-driven execution agent that autonomously plans action sequences, executes operations in a real Excel environment, and concurrently produces structured documentation and video demonstrations—without requiring human-provided step-by-step instructions or exemplar materials. Key contributions include: (1) the first fully automated Excel tutorial generation system; (2) a hybrid LLM-human evaluation framework for comprehensive quality assessment; and (3) empirical results demonstrating an 8.5% improvement in task execution success rate, tutorial readability and pedagogical effectiveness on par with expert-authored counterparts, and a 20× reduction in generation time—thereby validating the feasibility of large-scale, high-quality tutorial production.

Existing automated layout models fail to accommodate spreadsheet-specific constraints: their continuous coordinate modeling disregards the inherent discrete grid structure, and they neglect semantic relationships such as data dependencies and contextual links. Method: This paper formally defines the spreadsheet layout generation task for the first time and proposes a zero-shot, training-free approach leveraging multimodal large language models (MLLMs). It integrates rule-based reflection—ensuring alignment and non-overlap—with visual reflection—optimizing balance and readability—to overcome limitations of pure rectangular, continuous-space modeling. Contribution/Results: We introduce a seven-criterion evaluation protocol and validate our method on a dataset of 3,326 real-world spreadsheets. Compared to five baseline methods, our approach achieves ≥22.6% improvement in layout quality, generating structurally sound and semantically coherent complex layouts. Code and data are publicly released.

Existing PPT understanding benchmarks focus narrowly on isolated subtasks, overlooking the core challenge of joint visual-structural reasoning centered on layout. To address this gap, we introduce PPTBench—the first multimodal benchmark for PowerPoint layout and design understanding—constructed from 958 real-world PPTX files and comprising 4,439 samples with both visual inputs and JSON-structured annotations across four task categories: detection, comprehension, modification, and generation. Through systematic evaluation of state-of-the-art multimodal large language models (MLLMs), we uncover critical deficiencies in spatial relation modeling, precise element localization, and visual-semantic alignment—manifesting as misalignment and overlapping predictions—and demonstrate a strong correlation between layout-aware capability and API planning performance. This work fills a fundamental void in structured visual reasoning evaluation for slide-based content and establishes a new benchmark and analytical framework for joint visual-structural modeling.

This work addresses the challenges large language models face in generating accurate and readable statistical visualizations, stemming from existing datasets' lack of full alignment among code, data context, and question-answer pairs. The authors propose a structured, multi-stage workflow that decomposes chart generation into verifiable steps—data filtering, plotting proposal, code synthesis, rendering, and validation-driven refinement—and introduces a rendering feedback mechanism to transform the task from one-shot code generation into an iterative, verifiable process. The approach jointly produces charts, code, contextual metadata, natural language descriptions, and associated question-answer pairs, yielding a benchmark of 1,500 visualizations (spanning 24 chart types) and 30,003 QA pairs across 74 UCI datasets. Evaluation of 16 vision-language models demonstrates that this framework effectively exposes their limitations in numerical extraction, comparison, and reasoning, enabling fine-grained assessment of visual reasoning capabilities.

Data cleaning remains highly manual, inefficient, and error-prone. This paper proposes the first goal-driven LLM-based framework for automatic workflow generation: given a dirty table and a target query, it end-to-end generates a minimal viable clean table along with executable cleaning steps—including deduplication, missing-value imputation, and format standardization. Our contributions are threefold: (1) We introduce the first benchmark dataset comprising annotated quadruples of (goal, dirty table, cleaning workflow, cleaned answer); (2) We design a zero-shot, multi-stage prompting framework—requiring no fine-tuning—that decomposes the task into goal column identification, data quality diagnosis, and operation-parameter generation; (3) We empirically validate that off-the-shelf LLMs possess inherent reasoning capabilities sufficient to generate high-quality, executable cleaning workflows across three major LLM families, significantly reducing human intervention.

Existing agents lack fine-grained evaluation methodologies for complex, multimodal PowerPoint tasks, making it difficult to assess partial completion and diverse correct solutions. This work proposes the first fine-grained evaluation framework specifically designed for PowerPoint manipulation, introducing a benchmark comprising 120 tasks and a multidimensional scoring mechanism based on human-designed rubrics. The framework supports partial credit for content creation and editing tasks, incorporates aesthetic penalties, detects redundant operations, and provides natural language feedback. It achieves a Kendall’s τ-b correlation of 0.77 with human judgments. Experimental results reveal that even state-of-the-art models, such as Claude-4.5-Opus, attain only a 45% full-task success rate and a 57% average partial score, highlighting significant limitations in current agent capabilities.

This study investigates the mechanisms and pathways through which human domain knowledge is integrated into machine learning (ML) workflows via visual analytics. Building upon a systematic review of over 200 VIS4ML papers, the authors develop a coding framework encompassing four dimensions: machine learning, visualization, interaction, and action. By synthesizing perspectives from model construction and information-theoretic cost–benefit analysis, they propose the first unified explanatory framework for knowledge injection in ML. The work elucidates the pivotal role of interactive visualization in optimizing ML workflows and systematically maps the multidimensional pathways through which human expertise is incorporated. This contribution provides both theoretical grounding and empirical foundations for advancing research and practice in the VIS4ML community.

This work addresses the limitations of existing spreadsheet benchmarks, which focus on isolated operations and fail to evaluate end-to-end, multi-step workflows across worksheets as encountered in real-world business settings. To bridge this gap, the study introduces the first evaluation framework tailored to complex commercial spreadsheet workflows, constructing a realistic benchmark based on authentic data that encompasses generation, debugging, and visualization tasks—each involving an average of 11.8 worksheets and 593.5 cell modifications. The authors further develop an expert-annotated taxonomy for task decomposition and failure attribution. Evaluating state-of-the-art large language models and existing LLM-powered spreadsheet tools within a unified multi-turn agent architecture reveals that even the best-performing model achieves only 34.89% overall accuracy, with debugging tasks scoring as low as 12.00%, highlighting critical bottlenecks in target cell localization and structural understanding of spreadsheets.

This work addresses the significant limitations of spreadsheet-based analysis in reproducibility, auditability, version control, and automation. It proposes a migration pathway from Excel to research-grade analytical workflows by leveraging Python’s pandas library as a bridge. The study introduces an innovative set of Excel-to-pandas mapping rules, categorizes nine canonical workflow patterns, and compiles a catalog of common failure modes. Seven end-to-end real-world examples demonstrate the approach in practice. By retaining Excel as a familiar interface for input and output while integrating version control, automated refreshing, and seamless incorporation of statistical and machine learning methods, the proposed framework enables governed, reproducible, and auditable tabular data analysis.

This study investigates how to effectively leverage large language model (LLM) agents to generate scientific visualization workflows from natural language instructions, balancing performance, efficiency, and flexibility. We present the first systematic evaluation of three LLM agent paradigms—domain-specific, computer-use, and general-purpose programming agents—across 15 scientific visualization benchmark tasks. The analysis examines how interaction modalities (GUI/CLI/API/MCP/scripting), tool invocation strategies, and persistent memory mechanisms influence visualization quality, efficiency, and robustness. Results show that general-purpose programming agents achieve the highest success rates but incur substantial computational overhead; domain-specific agents are efficient yet inflexible; computer-use agents struggle with long-horizon planning; and persistent memory enhances performance on repeated tasks, though its efficacy depends critically on interaction modality and feedback quality. This work offers guidance for designing next-generation intelligent visualization systems that integrate multiple agent mechanisms.