Zero-Shot Size Transfer for Neural ODEs on Sparse Random Graphs: Graphon Limits and Adjoint Convergence

2026-06-25

AI Summary

This work investigates zero-shot transfer of Graph Neural Differential Equation (GNDE) models from small-scale sparse random graphs to larger yet structurally similar graphs without retraining. By constructing the infinite-node forward and adjoint systems of GNDE in the graphon limit, the study establishes the first rigorous trajectory-level convergence guarantees for cross-scale transfer on sparse graphs: the forward solution converges at a rate of $O((\alpha_n n)^{-1/2})$, and uniform convergence bounds are provided for the adjoint system with respect to time and parameter gradients. The analysis also reveals the asymptotic equivalence between DTO and OTD training strategies. Theoretical convergence rates are validated on HSBM and tent graphons, and successful zero-shot cross-scale deployment is demonstrated across four distinct graphon families.

Abstract

Graph Neural Differential Equations (GNDEs) model continuous-time graph dynamics by parameterizing Neural ODE velocity fields with Graph Neural Networks. Their local, size-independent filters suggest a zero-shot size-transfer principle: train on a small graph and deploy on larger, similar graphs without retraining. We develop a quantitative theory for this principle on sparse random graphs sampled from graphons. We consider Graphon Neural Differential Equations (Graphon-NDEs) and adjoint Graphon-NDEs as the infinite-node limits of the forward and adjoint GNDE systems, and establish well-posedness. For an $n$-node random graph with sparsity parameter $α_n$, we prove trajectory-wise convergence of GNDE solutions to Graphon-NDE solutions at rate $O((α_n n)^{-1/2})$, up to logarithmic factors, with high probability. We also establish uniform-in-time convergence bounds for adjoint systems governing hidden-state and parameter gradients. We further study discretize-then-optimize (DTO) and optimize-then-discretize (OTD) training. Under explicit Euler discretization with $M$ steps, we show that DTO and OTD are asymptotically consistent, with hidden-state and local parameter-gradient discrepancies of orders $O(1/M)$ and $O(1/M^2)$, respectively, up to sparsity and logarithmic factors. Experiments on HSBM and tent graphons support the theoretical rates, while zero-shot transfer experiments across four graphon classes demonstrate accurate deployment of learned GNDEs on larger independently sampled graphs.

Problem

• Zero-Shot Size Transfer
• Neural ODEs
• Sparse Random Graphs
• Graphon Limits
• Adjoint Convergence

Innovation

• Graph Neural Differential Equations
• Graphon Limits
• Zero-Shot Size Transfer
• Adjoint Convergence
• Sparse Random Graphs