DIVERSE: Disagreement-Inducing Vector Evolution for Rashomon Set Exploration

2026-01-28

AI Summary

This work investigates efficient exploration of the Rashomon set of deep neural networks—namely, generating ensembles of models that achieve accuracy comparable to a reference model while exhibiting diverse predictive behaviors. To this end, we propose the first integration of Feature-wise Linear Modulation (FiLM) with Covariance Matrix Adaptation Evolution Strategy (CMA-ES), enabling gradient-free, retraining-free search within the latent modulation space of a pretrained model. This approach substantially reduces computational overhead and successfully yields multiple high-accuracy model variants with significant functional diversity on MNIST, PneumoniaMNIST, and CIFAR-10, achieving diversity levels on par with methods that require full retraining.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Computer Vision: Multi-modal Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We propose DIVERSE, a framework for systematically exploring the Rashomon set of deep neural networks, the collection of models that match a reference model's accuracy while differing in their predictive behavior. DIVERSE augments a pretrained model with Feature-wise Linear Modulation (FiLM) layers and uses Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search a latent modulation space, generating diverse model variants without retraining or gradient access. Across MNIST, PneumoniaMNIST, and CIFAR-10, DIVERSE uncovers multiple high-performing yet functionally distinct models. Our experiments show that DIVERSE offers a competitive and efficient exploration of the Rashomon set, making it feasible to construct diverse sets that maintain robustness and performance while supporting well-balanced model multiplicity. While retraining remains the baseline to generate Rashomon sets, DIVERSE achieves comparable diversity at reduced computational cost.

Problem

• Rashomon set
• model diversity
• deep neural networks
• predictive behavior
• model multiplicity

Innovation

• Rashomon set
• FiLM
• CMA-ES
• model diversity
• zero-gradient exploration