Automated machine learning for physics-informed convolutional neural networks

2024-07-08

AI Summary

Physics-informed convolutional neural networks (PICNNs) suffer from heavy reliance on manual design, poor generalizability, and limited adaptability across diverse partial differential equation (PDE) problems. Method: We propose the first AutoML framework for PICNNs, featuring a two-stage neural architecture search (NAS) that jointly optimizes CNN architectures and physics-informed loss functions. Specifically, we construct a learnable loss-function factor space and residual adjustment operators, and integrate them with a physics-constrained CNN architecture search space. Contribution/Results: This work pioneers co-automated design of both network topology and physical loss components. Evaluated on multiple PDE benchmarks, our method achieves significantly faster convergence, higher prediction accuracy, and superior cross-problem generalization compared to handcrafted PICNNs—effectively addressing the long-standing challenge of coupled optimization between model architecture and physics-based loss functions in physics-driven learning.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Recent advances in deep learning for solving partial differential equations (PDEs) have introduced physics-informed neural networks (PINNs), which integrate machine learning with physical laws. Physics-informed convolutional neural networks (PICNNs) extend PINNs by leveraging CNNs for enhanced generalization and efficiency. However, current PICNNs depend on manual design, and inappropriate designs may not effectively solve PDEs. Furthermore, due to the diversity of physical problems, the ideal network architectures and loss functions vary across different PDEs. It is impractical to find the optimal PICNN architecture and loss function for each specific physical problem through extensive manual experimentation. To surmount these challenges, this paper uses automated machine learning (AutoML) to automatically and efficiently search for the loss functions and network architectures of PICNNs. We introduce novel search spaces for loss functions and network architectures and propose a two-stage search strategy. The first stage focuses on searching for factors and residual adjustment operations that influence the loss function, while the second stage aims to find the best CNN architecture. Experimental results show that our automatic searching method significantly outperforms the manually-designed model on multiple datasets.

Problem

• Automating loss function and architecture search for physics-informed CNNs
• Overcoming manual design limitations in solving diverse PDE problems
• Developing automated search strategy for optimal CNN architecture and loss functions

Innovation

• AutoML searches for optimal PICNN loss functions
• Two-stage strategy finds CNN architectures automatically
• Novel search spaces enhance physics-informed neural networks