Towards Deep Physics-Informed Kolmogorov-Arnold Networks

2025-10-27

AI Summary

To address the training instability and poor convergence of deep Chebyshev physics-informed Kolmogorov–Arnold networks (cPIKANs), this work proposes two core improvements: (1) a basis-function-agnostic Glorot-like initialization scheme, ensuring consistent parameter scaling across deep KAN layers; and (2) a residual gated adaptive architecture (RGA-KAN), integrating adaptive activation initialization with information bottleneck analysis to enhance gradient propagation and representational efficiency. Within a physics-informed machine learning (PIML) framework, the method incorporates Chebyshev polynomial expansions for enhanced spectral approximation. Evaluated on seven canonical PDE benchmarks, the proposed approach achieves stable convergence and delivers solution accuracy one to two orders of magnitude higher than parameter-matched cPIKANs and PirateNets. These advances significantly broaden the applicability and robustness of deep KANs for modeling complex partial differential equations.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Since their introduction, Kolmogorov-Arnold Networks (KANs) have been successfully applied across several domains, with physics-informed machine learning (PIML) emerging as one of the areas where they have thrived. In the PIML setting, Chebyshev-based physics-informed KANs (cPIKANs) have become the standard due to their computational efficiency. However, like their multilayer perceptron-based counterparts, cPIKANs face significant challenges when scaled to depth, leading to training instabilities that limit their applicability to several PDE problems. To address this, we propose a basis-agnostic, Glorot-like initialization scheme that preserves activation variance and yields substantial improvements in stability and accuracy over the default initialization of cPIKANs. Inspired by the PirateNet architecture, we further introduce Residual-Gated Adaptive KANs (RGA KANs), designed to mitigate divergence in deep cPIKANs where initialization alone is not sufficient. Through empirical tests and information bottleneck analysis, we show that RGA KANs successfully traverse all training phases, unlike baseline cPIKANs, which stagnate in the diffusion phase in specific PDE settings. Evaluations on seven standard forward PDE benchmarks under a fixed training pipeline with adaptive components demonstrate that RGA KANs consistently outperform parameter-matched cPIKANs and PirateNets - often by several orders of magnitude - while remaining stable in settings where the others diverge.

Problem

• Addressing training instabilities in deep physics-informed KANs
• Improving initialization schemes for deep neural networks
• Mitigating divergence issues in deep PDE solvers

Innovation

• Glorot-like initialization scheme for stable deep KANs
• Residual-Gated Adaptive KANs to mitigate divergence
• Basis-agnostic approach enhancing accuracy and stability