SAS: Semantic-aware Sampling for Generative Dataset Distillation

2026-05-18

AI Summary

This work addresses the limitation of existing dataset distillation methods, which often overlook high-level semantic information and struggle to balance class discriminability with sample diversity. To overcome this, the authors propose a semantic-aware dataset distillation framework that leverages CLIP as a semantic prior for the first time. They introduce three semantic scoring functions and a two-stage sampling strategy: first selecting samples with strong semantic discriminability, then dynamically choosing diverse instances to minimize redundancy. This approach systematically integrates class relevance, inter-class separability, and intra-set diversity in the semantic space, establishing a semantic-driven criterion for efficient dataset compression. Extensive experiments across multiple datasets, image pools, and downstream models demonstrate that the proposed method consistently outperforms current state-of-the-art approaches, confirming the effectiveness and generalizability of incorporating semantic information into dataset distillation.

Abstract

Deep neural networks have achieved impressive performance across a wide range of tasks, but this success often comes with substantial computational and storage costs due to large-scale training data. Dataset distillation addresses this challenge by constructing compact yet informative datasets that enable efficient model training while maintaining downstream performance. However, most existing approaches primarily emphasize matching data distributions or downstream training statistics, with limited attention to preserving high-level semantic information in the distilled data. In this work, we introduce a semantic-aware perspective for dataset distillation by leveraging Contrastive Language-Image Pretraining (CLIP) as a semantic prior for post-sampling. Our goal is to obtain distilled datasets that are not only compact but also semantically class-discriminative and diverse. To this end, we design three semantic scoring functions that quantify class relevance, inter-class separability, and intra-set diversity in a pretrained semantic space. Based on image pools generated by existing distillation methods, we further develop a two-stage strategy for effective sampling: the first stage filters semantically discriminative samples to form a reliable candidate set, and the second stage performs a dynamic diversity-aware selection to reduce redundancy while preserving semantic coverage. Extensive experiments across multiple datasets, image pools, and downstream models demonstrate consistent performance gains, highlighting the effectiveness of incorporating semantic information into dataset distillation.

Problem

• dataset distillation
• semantic information
• class-discriminative
• diversity
• compact dataset

Innovation

• semantic-aware sampling
• dataset distillation
• CLIP
• diversity-aware selection
• semantic scoring