Efficient Uncertainty Estimation for LLM-based Entity Linking in Tabular Data

2025-09-24

AI Summary

Large language models (LLMs) incur high computational overhead in table entity linking (EL) due to reliance on multi-step reasoning for uncertainty estimation. Method: This paper proposes an efficient, single-forward-pass uncertainty quantification method that constructs a self-supervised learning framework to extract discriminative features from token-level hidden states of the LLM’s one-time output, directly modeling prediction confidence—without introducing auxiliary parameters, fine-tuning, or modifying the original inference pipeline. Contribution/Results: Evaluated across multiple state-of-the-art LLMs (e.g., Llama-3, Qwen2) and standard EL benchmarks, the method achieves near-zero computational overhead while accurately identifying low-quality entity links. It significantly reduces the need for downstream error correction and re-inference, thereby enhancing both efficiency and cost-effectiveness of EL systems in real-world deployment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Linking textual values in tabular data to their corresponding entities in a Knowledge Base is a core task across a variety of data integration and enrichment applications. Although Large Language Models (LLMs) have shown State-of-The-Art performance in Entity Linking (EL) tasks, their deployment in real-world scenarios requires not only accurate predictions but also reliable uncertainty estimates, which require resource-demanding multi-shot inference, posing serious limits to their actual applicability. As a more efficient alternative, we investigate a self-supervised approach for estimating uncertainty from single-shot LLM outputs using token-level features, reducing the need for multiple generations. Evaluation is performed on an EL task on tabular data across multiple LLMs, showing that the resulting uncertainty estimates are highly effective in detecting low-accuracy outputs. This is achieved at a fraction of the computational cost, ultimately supporting a cost-effective integration of uncertainty measures into LLM-based EL workflows. The method offers a practical way to incorporate uncertainty estimation into EL workflows with limited computational overhead.

Problem

• Estimating uncertainty in LLM-based entity linking for tabular data
• Reducing computational cost of uncertainty estimation in entity linking
• Detecting low-accuracy outputs in knowledge base linking tasks

Innovation

• Self-supervised uncertainty estimation from single-shot outputs
• Token-level features reduce multi-generation computational cost
• Cost-effective uncertainty integration for LLM-based entity linking