ProgramTab: Boosting Table Reasoning of LLMs via Programmatic Paradigm

2026-07-13

AI Summary

This work addresses the limitations of large language models (LLMs) in handling large or unstructured tables—such as those found on web pages—stemming from constraints in input length, long-context modeling, and mathematical or logical reasoning. To overcome these challenges, the authors propose ProgramTab, a novel framework that synergistically integrates programmatic and SQL-based paradigms. Specifically, ProgramTab leverages in-context learning to prompt LLMs to generate Python code for table preprocessing, followed by row- and column-level information extraction and SQL query generation to accurately retrieve key content. By transcending the boundaries of purely textual or purely SQL-based approaches, ProgramTab achieves substantial performance gains over existing LLM baselines across multiple tabular reasoning benchmarks, demonstrating both its effectiveness and robustness.

Abstract

Table-based reasoning with large language models (LLMs), which requires reasoning based on natural language questions and structured tabular data, has gained widespread attention. However, a series of issues still constrain the application of this task. The previous approaches suffered from significant performance degradation when faced with large tables due to the difficulty of long text modeling and the limitation of input length for LLMs. The text-to-SQL approach is used to efficiently extract key information from tables and generate smaller sub-tables. However, tabular data, especially web tables, often lack the necessary structure and consistency, making them unsuitable for performing mathematical logic operations using SQL queries. We propose the ProgramTab framework, which guides LLMs employing in-context learning to perform tabular data preprocessing with Python code, as well as the momentous contents extraction with row and column extraction and SQL generation. The experiment results on table reasoning datasets demonstrate that the ProgramTab framework effectively deals with table-based reasoning tasks and outperforms all LLM-based baselines.

Problem

• table-based reasoning
• large language models
• structured tabular data
• input length limitation
• data preprocessing

Innovation

• ProgramTab
• table reasoning
• large language models
• programmatic paradigm
• in-context learning