Permutation-Invariant Spectral Learning via Dyson Diffusion

2025-10-09

AI Summary

Existing graph diffusion models suffer from limited discriminative capability across specific graph families due to architecture-imposed inductive biases, primarily stemming from the non-uniqueness of node orderings in adjacency matrices. Method: We propose a spectral learning framework that shifts inductive bias from network architecture to diffusion dynamics. For the first time, we integrate random matrix theory with Dyson Brownian motion to analytically model the spectral evolution of adjacency matrices. Our approach employs Dyson diffusion—formulated as an Ornstein–Uhlenbeck process—combined with spectral decomposition and permutation-invariance constraints for generative modeling. Contribution/Results: The method strictly enforces permutation-invariant spectral representations while preserving non-spectral structural information. Experiments demonstrate substantial improvements in spectral structure modeling accuracy, achieving superior performance over state-of-the-art graph diffusion methods on canonical graph family discrimination tasks.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Computer Vision: Diffusion Models for Vision
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Diffusion models are central to generative modeling and have been adapted to graphs by diffusing adjacency matrix representations. The challenge of having up to $n!$ such representations for graphs with $n$ nodes is only partially mitigated by using permutation-equivariant learning architectures. Despite their computational efficiency, existing graph diffusion models struggle to distinguish certain graph families, unless graph data are augmented with ad hoc features. This shortcoming stems from enforcing the inductive bias within the learning architecture. In this work, we leverage random matrix theory to analytically extract the spectral properties of the diffusion process, allowing us to push the inductive bias from the architecture into the dynamics. Building on this, we introduce the Dyson Diffusion Model, which employs Dyson's Brownian Motion to capture the spectral dynamics of an Ornstein-Uhlenbeck process on the adjacency matrix while retaining all non-spectral information. We demonstrate that the Dyson Diffusion Model learns graph spectra accurately and outperforms existing graph diffusion models.

Problem

• Overcoming permutation invariance limitations in graph diffusion models
• Enhancing graph family discrimination without ad hoc features
• Accurately learning graph spectral properties through diffusion dynamics

Innovation

• Leverages random matrix theory for spectral analysis
• Introduces Dyson Diffusion Model with Brownian Motion
• Shifts inductive bias from architecture to dynamics