MAGIC: Near-Optimal Data Attribution for Deep Learning

2025-04-23

AI Summary

Predicting data attribution aims to quantify how the addition or removal of individual training samples affects model predictions. In large-scale, non-convex deep learning settings, existing methods yield estimates that exhibit extremely low correlation with the true perturbation effects. This paper introduces the first attribution framework unifying infinitesimal Jackknife with metadifferentiation. By leveraging Hessian-vector product approximations and efficient gradient backpropagation, our method achieves near-optimal estimation of data perturbation effects. Empirically, it significantly improves attribution accuracy: across multiple deep architectures and benchmark datasets, the correlation between attribution scores and ground-truth perturbation effects increases markedly—approaching the theoretical optimal lower bound. Our approach establishes a new paradigm for precise data attribution in large-scale non-convex optimization settings.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The goal of predictive data attribution is to estimate how adding or removing a given set of training datapoints will affect model predictions. In convex settings, this goal is straightforward (i.e., via the infinitesimal jackknife). In large-scale (non-convex) settings, however, existing methods are far less successful -- current methods' estimates often only weakly correlate with ground truth. In this work, we present a new data attribution method (MAGIC) that combines classical methods and recent advances in metadifferentiation to (nearly) optimally estimate the effect of adding or removing training data on model predictions.

Problem

• Estimates impact of training data changes on model predictions
• Addresses weak correlation of current methods with ground truth
• Combines classical and modern techniques for optimal attribution

Innovation

• Combines classical and metadifferentiation methods
• Estimates training data impact on predictions
• Nearly optimal for non-convex deep learning