MODE: Multi-Objective Adaptive Coreset Selection

2025-12-24

AI Summary

Conventional static core-set selection fails to adapt to the heterogeneous requirements across different training stages. Method: This paper proposes a dynamic multi-objective adaptive core-set selection framework that dynamically switches sampling strategies according to training progression—emphasizing class balance in early stages, feature diversity in mid-stages, and prediction uncertainty in late stages—thereby enabling the first training-process-aware, multi-objective co-optimization. Contribution/Results: We theoretically establish a (1−1/e)-approximation guarantee. By integrating submodular optimization, active learning, and representation analysis, our method achieves O(n log n) computational efficiency. Empirically, it attains full-dataset accuracy on multiple benchmarks while significantly reducing memory overhead. Moreover, it is the first work to quantitatively characterize the dynamic evolution of data utility throughout training.

Technology Category

• Machine Learning: Active Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We present Mode(Multi-Objective adaptive Data Efficiency), a framework that dynamically combines coreset selection strategies based on their evolving contribution to model performance. Unlike static methods, mode adapts selection criteria to training phases: emphasizing class balance early, diversity during representation learning, and uncertainty at convergence. We show that MODE achieves (1-1/e)-approximation with O(n log n) complexity and demonstrates competitive accuracy while providing interpretable insights into data utility evolution. Experiments show mode reduces memory requirements

Problem

• Dynamically combines coreset selection strategies for model performance
• Adapts selection criteria to different training phases
• Reduces memory requirements while maintaining competitive accuracy

Innovation

• Dynamic multi-objective coreset selection
• Adaptive criteria across training phases
• Efficient approximation with interpretable insights