Enhancing Physics-Informed Neural Networks Through Feature Engineering

2025-02-11

AI Summary

To address the slow training, low accuracy, and ill-conditioned optimization commonly encountered when solving partial differential equations (PDEs) with physics-informed neural networks (PINNs), this paper proposes SAFE-NET—a single-layer adaptive feature engineering network. Our method explicitly incorporates Fourier feature encoding to embed physical priors, adopts a lightweight single-hidden-layer architecture to drastically reduce parameter count, and introduces a condition-aware optimization strategy that significantly improves the condition number of the loss function’s Hessian matrix. Experimental results across diverse PDE benchmarks demonstrate that SAFE-NET achieves 1–3 orders-of-magnitude reduction in prediction error, reduces model parameters by 65%, shortens training epochs to less than 30% of those required by baseline PINNs, and accelerates per-iteration computation by 95%. The approach thus delivers superior accuracy, computational efficiency, and generalization capability simultaneously.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Feature Construction/Reformulation
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Physics-Informed Neural Networks (PINNs) seek to solve partial differential equations (PDEs) with deep learning. Mainstream approaches that deploy fully-connected multi-layer deep learning architectures require prolonged training to achieve even moderate accuracy, while recent work on feature engineering allows higher accuracy and faster convergence. This paper introduces SAFE-NET, a Single-layered Adaptive Feature Engineering NETwork that achieves orders-of-magnitude lower errors with far fewer parameters than baseline feature engineering methods. SAFE-NET returns to basic ideas in machine learning, using Fourier features, a simplified single hidden layer network architecture, and an effective optimizer that improves the conditioning of the PINN optimization problem. Numerical results show that SAFE-NET converges faster and typically outperforms deeper networks and more complex architectures. It consistently uses fewer parameters -- on average, 65% fewer than the competing feature engineering methods -- while achieving comparable accuracy in less than 30% of the training epochs. Moreover, each SAFE-NET epoch is 95% faster than those of competing feature engineering approaches. These findings challenge the prevailing belief that modern PINNs effectively learn features in these scientific applications and highlight the efficiency gains possible through feature engineering.

Problem

• Improves PINNs accuracy
• Reduces training time
• Minimizes network parameters

Innovation

• Single-layered Adaptive Feature Engineering
• Fourier features optimization
• Reduced parameters, faster convergence