Generative Retrieval for Table Union Search

2026-07-01

AI Summary

This work addresses the limitations of traditional table union search—such as constrained recall, high latency, and substantial storage overhead—by introducing generative retrieval to the task for the first time. The approach reframes table union search as a constrained generation problem over discrete semantic table identifiers, enabling an end-to-end generator to directly produce compact identifiers of joinable tables, thereby eliminating explicit candidate retrieval and re-ranking stages. By integrating semantic-aware identifier design, generative training, and constrained decoding, the method achieves an average rank of 1.05 (best possible) across seven public benchmarks, significantly outperforming the strongest baseline (2.57), while substantially reducing online latency, storage requirements, and incremental update costs.

Abstract

Modern data lakes contain heterogeneous tables whose task-relevant information is often scattered across different schemas, sources, and naming conventions. Table union search (TUS) retrieves tables that can be reliably unioned with a query table, supporting data discovery, enrichment, and downstream analytics. Although learning-based TUS methods improve table- or column-level representations, they still follow an encode-search-refine pipeline: candidate retrieval is followed by query-candidate matching or reranking, making quality dependent on candidate-pool recall and incurring growing latency and storage costs as the data lake scales. We propose GenTUS, a generative retrieval framework that reformulates TUS as constrained generation over discrete semantic table identifiers. Instead of searching and reranking an explicit candidate pool, GenTUS assigns candidate tables compact unionability-aware identifiers and trains a generator to produce the identifiers of unionable tables directly from the query. At query time, constrained decoding ensures that generated identifiers correspond to valid data-lake tables and returns them as ranked retrieval results. Experiments on seven public TUS benchmarks show that GenTUS achieves the best overall retrieval quality, with an average rank of 1.05 compared to 2.57 for the strongest baseline, while substantially reducing online latency, retrieval-artifact storage, and incremental update cost.

Problem

• Table Union Search
• Data Lakes
• Generative Retrieval
• Heterogeneous Tables
• Candidate Retrieval

Innovation

• Generative Retrieval
• Table Union Search
• Constrained Decoding
• Semantic Identifiers
• Data Lakes