SketchFill: Sketch-Guided Code Generation for Imputing Derived Missing Values

2024-12-26

AI Summary

Numerical missing value imputation in tabular data requires complex arithmetic computations and modeling of cross-row/column relationships—a challenge inadequately addressed by existing methods. Method: This paper proposes a sketch-guided large language model (LLM) code generation paradigm that reformulates imputation as executable formula synthesis. It introduces a structured symbolic sketch prompting mechanism that explicitly encodes row/column mathematical dependencies and domain-specific constraints, overcoming the limitations of conventional chain-of-thought (CoT) reasoning in higher-order tabular inference. The approach integrates table semantic parsing, dynamic context distillation, and LLM-based program synthesis. Contribution/Results: Evaluated under derived missingness scenarios, the method achieves a 56.2% accuracy improvement over CoT and a 78.8% gain over MetaGPT, establishing new state-of-the-art performance for numerical imputation.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Missing value is a critical issue in data science, significantly impacting the reliability of analyses and predictions. Missing value imputation (MVI) is a longstanding problem because it highly relies on domain knowledge. Large language models (LLMs) have emerged as a promising tool for data cleaning, including MVI for tabular data, offering advanced capabilities for understanding and generating content. However, despite their promise, existing LLM techniques such as in-context learning and Chain-of-Thought (CoT) often fall short in guiding LLMs to perform complex reasoning for MVI, particularly when imputing derived missing values, which require mathematical formulas and data relationships across rows and columns. This gap underscores the need for further advancements in LLM methodologies to enhance their reasoning capabilities for more reliable imputation outcomes. To fill this gap, we propose SketchFill, a novel sketch-based method to guide LLMs in generating accurate formulas to impute missing numerical values. Our experimental results demonstrate that SketchFill significantly outperforms state-of-the-art methods, achieving 56.2% higher accuracy than CoT-based methods and 78.8% higher accuracy than MetaGPT. This sets a new standard for automated data cleaning and advances the field of MVI for numerical values.

Problem

• Large Language Models
• Data Analysis
• Missing Value Imputation

Innovation

• SketchFill
• Data Imputation
• Language Models