Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Space

2026-07-02

AI Summary

This work addresses the challenge of effectively modeling the high-dimensional weight space of neural networks and the inter-layer temporal dependencies inherent in their inference dynamics. To this end, the authors propose the Dynamic Neural Graph Encoder (DNG-Encoder), which, for the first time, formulates the layer-wise inference process of a neural network as a dynamic graph structure. By integrating Implicit Neural Representations (INRs), the method constructs a unified INR2JLS joint latent space tailored for downstream tasks such as INR classification. This approach transcends the limitations of conventional static weight analysis by leveraging dynamic graph neural networks to capture temporal patterns in deep weight spaces. Evaluated on the CIFAR-100-INR dataset, the proposed method achieves approximately a 10% improvement in classification accuracy over current state-of-the-art techniques, demonstrating its efficacy.

Abstract

The rapid advancements in using neural networks as implicit data representations have attracted significant interest in developing machine learning methods that analyze and process the weight spaces of other neural networks. However, efficiently handling these highdimensional weight spaces remains challenging. Existing methods often overlook the sequential nature of layer-by-layer processing in neural network inference. In this work, we propose a novel approach using dynamic graphs to represent neural network parameters, capturing the temporal dynamics of inference. Our Dynamic Neural Graph Encoder (DNG-Encoder) processes these graphs, preserving the sequential nature of neural processing. Additionally, we also leverage DNG-Encoder to develop INR2JLS (Implicit Neural Representation to Joint Latent Space) for facilitate downstream applications, such as classifying Implicit Neural Representations (INRs). Our approach demonstrates significant improvements across multiple tasks, surpassing the state-of-the-art INR classification accuracy by approximately 10% on the CIFAR-100-INR.

Problem

• weight space
• neural network inference
• sequential processing
• implicit neural representations
• high-dimensional representation

Innovation

• Dynamic Neural Graph
• Weight Space Encoding
• Implicit Neural Representation
• Sequential Inference Modeling
• INR Classification