OptiSQL: Executable SQL Generation from Optical Tokens

2026-01-20

AI Summary

This work proposes a vision-driven, end-to-end framework that directly generates executable SQL from tabular images paired with natural language questions, addressing the inefficiency and high token cost of conventional text-to-SQL methods when applied to visually presented tables. The approach employs an OCR-guided visual encoder to extract compact optical token representations of the table structure and freezes this encoder during training, enabling efficient fine-tuning of the decoder alone. This study demonstrates for the first time that optical representations can serve as an effective interface for semantic parsing. Evaluated on the Spider 2.0-Snow visual benchmark, the method achieves execution accuracy comparable to purely text-based approaches while reducing table input tokens by an order of magnitude and maintaining structural robustness under visual perturbations.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Search and Optimization: Learning to Search
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Executable SQL generation is typically studied in text-to-SQL settings, where tables are provided as fully linearized textual schemas and contents. While effective, this formulation assumes access to structured text and incurs substantial token overhead, which is misaligned with many real-world scenarios where tables appear as visual artifacts in documents or webpages. We investigate whether compact optical representations can serve as an efficient interface for executable semantic parsing. We present OptiSQL, a vision-driven framework that generates executable SQL directly from table images and natural language questions using compact optical tokens. OptiSQL leverages an OCR-oriented visual encoder to compress table structure and content into a small set of optical tokens and fine-tunes a pretrained decoder for SQL generation while freezing the encoder to isolate representation sufficiency. Experiments on a visualized version of Spider 2.0-Snow show that OptiSQL retains strong execution accuracy while reducing table input tokens by an order of magnitude. Robustness analyses further demonstrate that optical tokens preserve essential structural information under visual perturbations.

Problem

• executable SQL generation
• optical tokens
• table images
• semantic parsing
• vision-to-SQL

Innovation

• OptiSQL
• optical tokens
• vision-driven SQL generation
• executable semantic parsing
• OCR-oriented visual encoder