NTK-Guided Implicit Neural Teaching

2025-11-19

AI Summary

To address the high computational cost and slow convergence in training implicit neural representations (INRs) for high-resolution signal modeling—stemming from inefficient coordinate sampling—this paper proposes a neural tangent kernel (NTK)-guided dynamic coordinate selection mechanism. Our method quantifies sample importance via the NTK-calibrated norm of the loss gradient, jointly accounting for reconstruction error and inter-coordinate coupling to adaptively select coordinates that contribute most to global function updates. Unlike fixed or heuristic sampling strategies, our approach significantly improves training efficiency under standard MLP architectures. Experiments demonstrate an average 47% reduction in training time while maintaining or even improving reconstruction quality, establishing it as the new state-of-the-art (SOTA) among sampling-based INR acceleration methods.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Implicit Neural Representations (INRs) parameterize continuous signals via multilayer perceptrons (MLPs), enabling compact, resolution-independent modeling for tasks like image, audio, and 3D reconstruction. However, fitting high-resolution signals demands optimizing over millions of coordinates, incurring prohibitive computational costs. To address it, we propose NTK-Guided Implicit Neural Teaching (NINT), which accelerates training by dynamically selecting coordinates that maximize global functional updates. Leveraging the Neural Tangent Kernel (NTK), NINT scores examples by the norm of their NTK-augmented loss gradients, capturing both fitting errors and heterogeneous leverage (self-influence and cross-coordinate coupling). This dual consideration enables faster convergence compared to existing methods. Through extensive experiments, we demonstrate that NINT significantly reduces training time by nearly half while maintaining or improving representation quality, establishing state-of-the-art acceleration among recent sampling-based strategies.

Problem

• Accelerating training of Implicit Neural Representations for high-resolution signals
• Reducing computational costs by optimizing coordinate selection dynamically
• Maintaining representation quality while achieving faster convergence rates

Innovation

• NTK-Guided Implicit Neural Teaching accelerates training
• Dynamically selects coordinates maximizing functional updates
• Uses Neural Tangent Kernel to score loss gradients