Topological Neural Tangent Kernel

2026-05-01

AI Summary

Existing graph neural tangent kernels model only pairwise relationships, limiting their ability to capture higher-order interactions and topological structures. This work introduces Hodge theory into neural tangent kernels for the first time, proposing an infinite-width kernel method tailored to edge features on simplicial complexes. By jointly modeling vertex-sharing and filled-simplex coupling through upper and lower Hodge interactions, the approach enables topology-aware learning of higher-order structures. Grounded in Hodge decomposition, the method reveals interpretable learning dynamics associated with gradient, harmonic, and curl components, integrating simplicial message passing, spectral analysis, and stability theory. Experiments on synthetic tasks and high-order link prediction on DBLP demonstrate substantial improvements in expressivity, learning efficiency, and predictive performance.

Abstract

Graph neural tangent kernels give a principled infinite-width theory for graph neural networks, but inherit a basic limitation of graph models: they see only pairwise structure. Many relational systems contain higher-order interactions that are more naturally represented by simplicial complexes. We introduce the Topological Neural Tangent Kernel (TopoNTK), an infinite-width kernel for simplicial message passing on edge features. TopoNTK combines lower Hodge interactions, capturing graph-like coupling through shared vertices, with upper Hodge interactions, capturing coupling through filled simplices. This makes the kernel sensitive to topology invisible to graph kernels, allowing complexes with the same graph but different filled simplices to induce different kernels. Beyond expressivity, the Hodge structure gives the kernel an interpretable learning geometry. Edge signals decompose into gradient-like, harmonic, and local circulation components, and the spectrum of the TopoNTK determines how quickly each component is learned. This yields a topological form of spectral bias: components aligned with large-eigenvalue modes are learned quickly, while global harmonic modes, retained through the residual channel, often lie at smaller eigenvalues and are learned more slowly. We prove expressivity, Hodge-alignment, spectral learning, and stability properties, and validate them on synthetic simplicial tasks and DBLP higher-order link prediction. The results show that topology is not merely extra structure; it can provide coordinates that make relational learning more faithful, interpretable, and effective.

Problem

• simplicial complexes
• higher-order interactions
• topological structure
• graph neural tangent kernel
• Hodge theory

Innovation

• Topological Neural Tangent Kernel
• Simplicial Complexes
• Hodge Decomposition
• Higher-order Interactions
• Spectral Bias