Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries

2025-02-04

AI Summary

Existing neural PDE solvers are limited to tens of thousands of grid points, rendering them inadequate for industrial-scale simulations involving millions of degrees of freedom and complex geometries. To address this, we propose the first neural PDE solver capable of handling million-node inputs on a single GPU. Our method integrates a Transformer-based physical state learning model, a novel local adaptive attention mechanism, and a distributed tensor-parallel framework with memory optimization—ensuring both physical fidelity and linearly scalable multi-GPU parallelism. Evaluated on six standard PDE benchmarks, our approach reduces relative error by 13% compared to prior neural solvers. Moreover, it achieves over 20% performance improvement on large-scale industrial simulations—including automotive and full 3D aircraft models—demonstrating, for the first time, the practical feasibility and computational efficiency of neural PDE solvers in real-world industrial applications.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Although deep models have been widely explored in solving partial differential equations (PDEs), previous works are primarily limited to data only with up to tens of thousands of mesh points, far from the million-point scale required by industrial simulations that involve complex geometries. In the spirit of advancing neural PDE solvers to real industrial applications, we present Transolver++, a highly parallel and efficient neural solver that can accurately solve PDEs on million-scale geometries. Building upon previous advancements in solving PDEs by learning physical states via Transolver, Transolver++ is further equipped with an extremely optimized parallelism framework and a local adaptive mechanism to efficiently capture eidetic physical states from massive mesh points, successfully tackling the thorny challenges in computation and physics learning when scaling up input mesh size. Transolver++ increases the single-GPU input capacity to million-scale points for the first time and is capable of continuously scaling input size in linear complexity by increasing GPUs. Experimentally, Transolver++ yields 13% relative promotion across six standard PDE benchmarks and achieves over 20% performance gain in million-scale high-fidelity industrial simulations, whose sizes are 100$ imes$ larger than previous benchmarks, covering car and 3D aircraft designs.

Problem

• Neural solver for million-scale PDEs
• Efficient parallelism for complex geometries
• Linear scalability with GPU increase

Innovation

• Neural solver for million-scale PDEs
• Optimized parallelism framework
• Local adaptive mechanism for physical states