Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers

2024-05-27

arXiv.org

AI Summary

Existing neural PDE solvers exhibit poor generalization, being restricted to narrow classes of equations and coefficient functions, thus failing to serve as practical alternatives to traditional numerical methods. Method: We propose UniSolver, the first universal PDE solver built upon a componentized conditional modeling framework. It systematically encodes structural elements—such as governing equations (symbolic forms), spatially varying coefficients, and boundary conditions—into domain-level and point-level conditioning signals, fed into a physics-informed Transformer architecture. Multi-granularity conditional encoding and training on large-scale, heterogeneous PDE datasets enable unified modeling across diverse equations, coefficients, and boundary configurations. Contribution/Results: UniSolver achieves state-of-the-art performance across three challenging, high-fidelity benchmarks. Crucially, it demonstrates significantly improved cross-equation generalization—validating its viability as a robust, general-purpose numerical solver replacement.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Solvers and Tools

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Deep models have recently emerged as a promising tool to solve partial differential equations (PDEs), known as neural PDE solvers. While neural solvers trained from either simulation data or physics-informed loss can solve PDEs reasonably well, they are mainly restricted to a few instances of PDEs, e.g. a certain equation with a limited set of coefficients. This limits the generalization of neural solvers to diverse PDEs, impeding them from being practical surrogate models for numerical solvers. In this paper, we present the Universal PDE Solver (Unisolver) capable of solving a wide scope of PDEs by training a novel Transformer model on diverse data and conditioned on diverse PDEs. Instead of purely scaling up data and parameters, Unisolver stems from the theoretical analysis of the PDE-solving process. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components, e.g. equation symbols, coefficients, and boundary conditions. Inspired by the mathematical structure of PDEs, we define a complete set of PDE components and flexibly embed them as domain-wise (e.g. equation symbols) and point-wise (e.g. boundaries) conditions for Transformer PDE solvers. Integrating physical insights with recent Transformer advances, Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks, showing impressive performance gains and favorable PDE generalizability.

Problem

• Develops a universal neural PDE solver for diverse equations
• Overcomes limitations of existing neural solvers' generalization
• Integrates PDE components as deep conditions in Transformers

Innovation

• Transformer model conditioned on diverse PDEs
• Embedding PDE components as deep conditions
• Combining physical insights with Transformer advances