Better Think with Tables: Leveraging Tables to Enhance Large Language Model Comprehension

2024-12-22

arXiv.org

AI Summary

Large language models (LLMs) exhibit inaccurate reasoning and poor robustness when handling complex, multi-condition data analysis queries—such as multi-constraint retrieval and transformation tasks. Method: This paper proposes *Tabular Thinking*, a novel reasoning paradigm that employs a pre-instruction-guided, tabular intermediate representation to explicitly structure multi-step reasoning into rows and columns, aligning with human structured cognition. Contribution/Results: We formally define and empirically validate four levels of data structuralization and their impact on model performance; further, we introduce a multi-level structured evaluation framework. Experiments demonstrate that Tabular Thinking achieves an average relative performance improvement of 40.29% on multi-condition query tasks, significantly enhancing generalization and stability across diverse requests, constraints, and application scenarios.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Despite the recent advancement of Large Langauge Models (LLMs), they struggle with complex queries often involving multiple conditions, common in real-world scenarios. We propose Thinking with Tables, a technique that assists LLMs to leverage tables for intermediate thinking aligning with human cognitive behavior. By introducing a pre-instruction that triggers an LLM to organize information in tables, our approach achieves a 40.29% average relative performance increase, higher robustness, and show generalizability to different requests, conditions, or scenarios. We additionally show the influence of data structuredness for the model by comparing results from four distinct structuring levels that we introduce.

Problem

• Enhance LLM comprehension for data-analytics requests
• Improve information retrieval and data manipulation in LLMs
• Evaluate tabular structures for efficiency and effectiveness

Innovation

• Inject tabular structures into LLMs
• Tabular structures enhance information attention
• Text-to-table conversion for data analytics