SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation

2026-04-27

AI Summary

Existing large language model (LLM)-based tabular data generation methods often introduce spurious correlations due to dense modeling and assume static feature dependencies, failing to capture the dynamic, value-level dependencies prevalent in real-world data. To address this, this work proposes SAGE, a novel framework that first discretizes continuous features into value-aware pseudo-features, then constructs a sparse dependency graph based on mutual information, and dynamically guides the generation process through either explicit context selection or implicit logit correction. SAGE introduces, for the first time, a sparse and value-adaptive dependency modeling mechanism that enables context-sensitive control over LLM-based generation. Experiments across six datasets demonstrate that SAGE significantly improves synthetic data fidelity and downstream task performance, achieving an average 10% gain in F1 score over existing LLM-based methods while reducing policy violation rates by one percentage point.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Generating high-fidelity synthetic tabular data remains a critical challenge for enhancing data availability in privacy-sensitive and low-resource domains. Recent approaches leverage LLMs by representing table rows as sequences, yet suffer from two fundamental limitations: (1) they model feature dependencies densely, introducing spurious correlations; and (2) they assume static relationships between features, ignoring how these dependencies vary with feature values. To overcome these limitations, we introduce SAGE (Sparse Adaptive Guidance), a novel LLM-based generation framework that enforces sparse and dynamic dependency guidance. SAGE discretizes features into value-aware pseudo-features and constructs a mutual information-based sparse dependency graph. This graph adaptively guides generation through explicit context selection or implicit logit correction, enabling LLMs to focus on truly relevant information during synthesis. Our extensive experiments across six datasets and multiple tasks reveal that SAGE not only improves data fidelity and downstream utility, boosting F1 scores by 10% compared to previous LLM-based methods, but also reduces policy violations by one point. These results highlight the importance of adaptive structure in tabular data generation and provide new insights into context-sensitive control of LLMs.

Problem

• tabular data generation
• feature dependencies
• spurious correlations
• static relationships
• LLM-based generation

Innovation

• Sparse Dependency
• Adaptive Guidance
• Tabular Data Generation
• Mutual Information
• LLM-based Synthesis