Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data

2025-10-27

AI Summary

Conventional wisdom holds that the accuracy of PDE neural operators is fundamentally bounded by the numerical fidelity of their training data. This work challenges that assumption, empirically demonstrating the “surrogate superiority” phenomenon: neural surrogates trained exclusively on low-fidelity numerical solutions can outperform both their training data source and even the solver that generated that data—particularly in multi-step temporal evolution. Methodologically, we employ standard neural operator architectures, using low-fidelity PDE solutions as inputs, and evaluate long-term predictive accuracy against high-fidelity reference solutions via multi-step rollout. Experiments across diverse PDEs—including Navier–Stokes and Burgers equations—consistently yield higher-accuracy predictions. Our key contribution is revealing that neural networks, through implicit inductive biases and error-dynamics compensation mechanisms, can learn more accurate dynamical representations than those encoded in the training data, thereby transcending data fidelity limitations. This provides a novel theoretical perspective and practical pathway for data-driven PDE modeling.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Neural operators or emulators for PDEs trained on data from numerical solvers are conventionally assumed to be limited by their training data's fidelity. We challenge this assumption by identifying "emulator superiority," where neural networks trained purely on low-fidelity solver data can achieve higher accuracy than those solvers when evaluated against a higher-fidelity reference. Our theoretical analysis reveals how the interplay between emulator inductive biases, training objectives, and numerical error characteristics enables superior performance during multi-step rollouts. We empirically validate this finding across different PDEs using standard neural architectures, demonstrating that emulators can implicitly learn dynamics that are more regularized or exhibit more favorable error accumulation properties than their training data, potentially surpassing training data limitations and mitigating numerical artifacts. This work prompts a re-evaluation of emulator benchmarking, suggesting neural emulators might achieve greater physical fidelity than their training source within specific operational regimes. Project Page: https://tum-pbs.github.io/emulator-superiority

Problem

• Neural emulators surpass training data accuracy in PDE modeling
• Theoretical analysis reveals superior multi-step error accumulation properties
• Emulators implicitly learn more regularized dynamics than numerical solvers

Innovation

• Neural emulators surpass low-fidelity training data accuracy
• Theoretical analysis reveals inductive biases enable superior performance
• Emulators implicitly learn regularized dynamics mitigating numerical artifacts