UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation

2025-10-28

AI Summary

Existing generative data augmentation methods prioritize visual fidelity and diversity of synthetic data while neglecting its adaptability to downstream tasks and model architectures. This paper proposes UtilGen—the first task-utility-driven data augmentation framework—which dynamically optimizes the generation process via downstream task feedback, shifting the paradigm from “visual quality-oriented” to “task utility-oriented” augmentation. Its two-level optimization mechanism jointly refines model-level adaptation and instance-level generation strategies, integrating prompt embedding, initial noise modulation, and iterative reinforcement learning to co-optimize the generative model and a weight allocation network. Evaluated on eight benchmark datasets, UtilGen achieves an average accuracy improvement of 3.87% over state-of-the-art methods, demonstrating superior task relevance and training efficacy of the generated data.

Technology Category

• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Data augmentation using generative models has emerged as a powerful paradigm for enhancing performance in computer vision tasks. However, most existing augmentation approaches primarily focus on optimizing intrinsic data attributes -- such as fidelity and diversity -- to generate visually high-quality synthetic data, while often neglecting task-specific requirements. Yet, it is essential for data generators to account for the needs of downstream tasks, as training data requirements can vary significantly across different tasks and network architectures. To address these limitations, we propose UtilGen, a novel utility-centric data augmentation framework that adaptively optimizes the data generation process to produce task-specific, high-utility training data via downstream task feedback. Specifically, we first introduce a weight allocation network to evaluate the task-specific utility of each synthetic sample. Guided by these evaluations, UtilGen iteratively refines the data generation process using a dual-level optimization strategy to maximize the synthetic data utility: (1) model-level optimization tailors the generative model to the downstream task, and (2) instance-level optimization adjusts generation policies -- such as prompt embeddings and initial noise -- at each generation round. Extensive experiments on eight benchmark datasets of varying complexity and granularity demonstrate that UtilGen consistently achieves superior performance, with an average accuracy improvement of 3.87% over previous SOTA. Further analysis of data influence and distribution reveals that UtilGen produces more impactful and task-relevant synthetic data, validating the effectiveness of the paradigm shift from visual characteristics-centric to task utility-centric data augmentation.

Problem

• Optimizing data generation for task-specific utility requirements
• Adapting generative models through dual-level optimization strategy
• Shifting focus from visual fidelity to downstream task performance

Innovation

• Utility-centric generative data augmentation framework
• Dual-level optimization strategy for task adaptation
• Weight allocation network evaluates synthetic data utility