H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables

2024-06-29

arXiv.org

AI Summary

Table-based question answering (TableQA) faces challenges in jointly modeling natural language understanding and structured computation. Method: This paper proposes a two-stage hybrid reasoning framework: (1) a retrieval stage that identifies relevant cells via multi-perspective column selection and row-level contextual extraction; (2) a reasoning stage that dynamically routes queries—categorized as lookup, logical, or quantitative—to either semantic reasoning or SQL-enhanced symbolic reasoning paths. Contribution/Results: We introduce the first adaptive fusion mechanism integrating semantic and symbolic reasoning, enabling LLM-driven text comprehension and precise, verifiable computation in synergy. Our approach achieves significant improvements over state-of-the-art methods across three TableQA benchmarks and a fact verification dataset, demonstrating superior semantic depth, computational accuracy, and reasoning robustness.

Technology Category

• Natural Language Processing: Question Answering
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Tabular reasoning involves interpreting natural language queries about tabular data, which presents a unique challenge of combining language understanding with structured data analysis. Existing methods employ either textual reasoning, which excels in semantic interpretation but struggles with mathematical operations, or symbolic reasoning, which handles computations well but lacks semantic understanding. This paper introduces a novel algorithm H-STAR that integrates both symbolic and semantic (textual) approaches in a two-stage process to address these limitations. H-STAR employs: (1) step-wise table extraction using `multi-view' column retrieval followed by row extraction, and (2) adaptive reasoning that adapts reasoning strategies based on question types, utilizing semantic reasoning for direct lookup and complex lexical queries while augmenting textual reasoning with symbolic reasoning support for quantitative and logical tasks. Our extensive experiments demonstrate that H-STAR significantly outperforms state-of-the-art methods across three tabular question-answering (QA) and fact-verification datasets, underscoring its effectiveness and efficiency.

Problem

• Combining language understanding with structured data analysis
• Integrating symbolic and semantic reasoning for tabular queries
• Adapting reasoning strategies based on question types

Innovation

• Hybrid symbolic-semantic reasoning for tables
• Multi-view column and row extraction
• Adaptive reasoning based on question types