Model Reprogramming Demystified: A Neural Tangent Kernel Perspective

2025-05-31

AI Summary

This work addresses the lack of theoretical foundations for model reprogramming (MR). We establish, for the first time, a unified theoretical framework grounded in the Neural Tangent Kernel (NTK), revealing that MR’s success hinges on the spectral structure of the NTK evaluated on target data and the effective transfer mechanism of source-model capability. Theoretically, we prove that MR performance is governed by the NTK eigenvalue distribution and derive sufficient conditions for effective capability transfer from source to target models. Methodologically, we integrate NTK analysis, spectral theory, and the MR paradigm, validated empirically across diverse domains—including computer vision and time-series forecasting. Experiments demonstrate a strong correlation between the source model’s NTK spectral properties and MR efficacy, providing an interpretable, principled theoretical basis for MR and filling a critical gap in its theoretical understanding.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Model Reprogramming (MR) is a resource-efficient framework that adapts large pre-trained models to new tasks with minimal additional parameters and data, offering a promising solution to the challenges of training large models for diverse tasks. Despite its empirical success across various domains such as computer vision and time-series forecasting, the theoretical foundations of MR remain underexplored. In this paper, we present a comprehensive theoretical analysis of MR through the lens of the Neural Tangent Kernel (NTK) framework. We demonstrate that the success of MR is governed by the eigenvalue spectrum of the NTK matrix on the target dataset and establish the critical role of the source model's effectiveness in determining reprogramming outcomes. Our contributions include a novel theoretical framework for MR, insights into the relationship between source and target models, and extensive experiments validating our findings.

Problem

• Theoretical foundations of Model Reprogramming are underexplored
• Analyzes Model Reprogramming via Neural Tangent Kernel framework
• Examines source model's role in reprogramming outcomes

Innovation

• Uses Neural Tangent Kernel for analysis
• Links NTK eigenvalues to reprogramming success
• Validates with extensive experimental results