ModHiFi: Identifying High Fidelity predictive components for Model Modification

2025-11-24

AI Summary

Open-weight models pose challenges for component-level modification tasks (e.g., pruning or unlearning) when training data, loss functions, and gradient information are unavailable. Method: This paper proposes ModHiFi, an efficient, label-free, gradient-free importance estimation framework. Its core innovations are: (1) the Subset Fidelity metric, which quantifies global component importance via local reconstruction behavior—a first-of-its-kind formulation; and (2) a theoretical linkage between local and global reconstruction errors grounded in Lipschitz continuity, enabling fully unsupervised, data-free importance assessment. Results: ModHiFi-P achieves 11% higher speedup over state-of-the-art pruning methods on ImageNet. ModHiFi-U enables complete, zero-fine-tuning unlearning on CIFAR-10 and demonstrates strong generalization to Swin Transformers. Collectively, ModHiFi bridges a critical gap in model interpretability and editability under minimal supervision constraints.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Open weight models, which are ubiquitous, rarely provide access to their training data or loss function. This makes modifying such models for tasks such as pruning or unlearning constrained by this unavailability an active area of research. Existing techniques typically require gradients or ground-truth labels, rendering them infeasible in settings with limited computational resources. In this work, we investigate the fundamental question of identifying components that are critical to the model's predictive performance, without access to either gradients or the loss function, and with only distributional access such as synthetic data. We theoretically demonstrate that the global reconstruction error is linearly bounded by local reconstruction errors for Lipschitz-continuous networks such as CNNs and well-trained Transformers (which, contrary to existing literature, we find exhibit Lipschitz continuity). This motivates using the locally reconstructive behavior of component subsets to quantify their global importance, via a metric that we term Subset Fidelity. In the uncorrelated features setting, selecting individual components via their Subset Fidelity scores is optimal, which we use to propose ModHiFi, an algorithm for model modification that requires no training data or loss function access. ModHiFi-P, for structured pruning, achieves an 11% speedup over the current state of the art on ImageNet models and competitive performance on language models. ModHiFi-U, for classwise unlearning, achieves complete unlearning on CIFAR-10 without fine-tuning and demonstrates competitive performance on Swin Transformers.

Problem

• Identifying critical model components without gradients or loss function access
• Enabling model pruning and unlearning without training data requirements
• Developing subset fidelity metric to quantify component importance locally

Innovation

• Uses Subset Fidelity metric without gradients
• Identifies critical components via local reconstruction
• Enables pruning and unlearning without training data