When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors

2026-06-30

AI Summary

Large language models frequently suffer from reasoning failures in tabular tasks due to data reference errors (DREs). This work presents the first systematic evaluation of DRE behavior across diverse tabular tasks and multiple models, and introduces a lightweight 4B-parameter critic model trained end-to-end to effectively detect both in-distribution and out-of-distribution reference errors. By integrating rejection sampling and filtering mechanisms, the proposed critic significantly enhances downstream task accuracy. Evaluated across multiple models and tasks, the approach achieves an average F1 score of 78.2% and improves answer accuracy by up to 12.0%.

Abstract

While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.

Problem

• data referencing errors
• large language models
• table reasoning
• intermediate reasoning
• reliability

Innovation

• data referencing errors
• critic model
• table reasoning
• rejection sampling
• LLM evaluation