Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks

2026-04-16

AI Summary

This work addresses the challenges of slow convergence, training instability, and insufficient accuracy commonly encountered in Physics-Informed Neural Networks (PINNs), which stem from highly anisotropic and rapidly varying loss landscapes. To overcome these issues, the authors propose a lightweight, curvature-aware optimization framework that constructs an inexpensive surrogate for local geometric variations by leveraging secant curvature estimation and continuous gradient difference analysis. An adaptive correction mechanism with step-size normalization is introduced to enhance existing first-order optimizers, without requiring explicit computation of second-order matrices. The method is plug-and-play, computationally efficient, and broadly compatible with standard optimizers. Extensive experiments on high-dimensional and complex dynamical system benchmarks—including the heat equation, Gray–Scott, Belousov–Zhabotinsky, and Kuramoto–Sivashinsky systems—demonstrate significant improvements in convergence speed, training stability, and solution accuracy.

Technology Category

• Machine Learning: Optimization
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Physics-Informed Neural Networks (PINNs) often suffer from slow convergence, training instability, and reduced accuracy on challenging partial differential equations due to the anisotropic and rapidly varying geometry of their loss landscapes. We propose a lightweight curvature-aware optimization framework that augments existing first-order optimizers with an adaptive predictive correction based on secant information. Consecutive gradient differences are used as a cheap proxy for local geometric change, together with a step-normalized secant curvature indicator to control the correction strength. The framework is plug-and-play, computationally efficient, and broadly compatible with existing optimizers, without explicitly forming second-order matrices. Experiments on diverse PDE benchmarks show consistent improvements in convergence speed, training stability, and solution accuracy over standard optimizers and strong baselines, including on the high-dimensional heat equation, Gray--Scott system, Belousov--Zhabotinsky system, and 2D Kuramoto--Sivashinsky system.

Problem

• Physics-Informed Neural Networks
• loss landscape geometry
• training instability
• slow convergence
• partial differential equations

Innovation

• curvature-aware optimization
• physics-informed neural networks
• lightweight adaptation
• secant information
• loss landscape geometry