Post-Generation Curation of Synthetic Images via Homogeneous-Heterogeneous Splitting

2026-07-02

AI Summary

This work addresses a critical yet overlooked limitation of modern generative models: their tendency to produce class-typical samples at the expense of intra-class diversity, thereby diminishing the utility of synthetic data in downstream tasks. The study is the first to formally characterize this structural bias and introduces a novel post-hoc filtering mechanism that requires neither retraining nor generator-specific modifications. By partitioning real classes into homogeneous typical (HO) and heterogeneous non-redundant (HE) subsets, the method selects high-quality synthetic samples through a fidelity–diversity criterion that combines semantic alignment scores with redundancy penalties. Evaluated across multiple benchmarks, the approach consistently outperforms existing data selection strategies—achieving performance on par with real data using only 60% of the synthetic samples—and provides consistent gains even when applied to strong generative models in both classification and segmentation tasks.

Abstract

Recent generative models can produce high-quality synthetic images, offering scalable training training data for data-hungry models. Existing approaches to exploiting this potential typically involve 1) training or fine-tuning generators, or 2) using lightweight post-hoc adaptation like prompt engineering or inference-time guidance, making them generator-specific and expertise-intensive. We study a complementary question: given a fixed pool of generated images, can downstream utility be improved purely by selecting an informative subset? The answer is yes. We show that effective selection must counter a structural bias of modern generators: they tend to over-produce canonical modes of each class while underrepresenting intra-class variation. Building on this insight, we split each real class into a canonical Homogeneous (HO) subset and a non-redundant Heterogeneous (HE) subset, then score synthetic images by a fidelity-diversity criterion that rewards semantic alignment while penalizing canonical redundancy. The method is generator-agnostic and requires no retraining. Across multiple benchmarks, it consistently outperforms state-of-the-art data selection baselines and matches the real-data performance with up to 40% fewer synthetic samples. The same criterion remains effective when applied on top of stronger task-tuned generators, with gains on both classification and segmentation tasks. Post-generation selection is therefore not a substitute for better generators, but a complementary mechanism for improving the utility of synthetic data.

Problem

• synthetic data
• post-generation curation
• class diversity
• canonical bias
• data selection

Innovation

• synthetic data selection
• homogeneous-heterogeneous splitting
• fidelity-diversity trade-off
• generator-agnostic curation
• intra-class variation