Generating synthetic data for neural operators

2024-01-04

arXiv.org

AI Summary

Neural operator training is severely constrained by reliance on data generated via traditional numerical solvers, which are computationally expensive, discretization-dependent, and introduce approximation errors. Method: We propose a solver-free backward synthetic data generation framework: candidate solutions $u_j$ are randomly sampled from the solution space (e.g., $H_0^1(Omega)$), and their exact source terms $f_j = mathcal{L}u_j$ are computed directly via automatic differentiation—yielding infinite, zero-error $(f_j, u_j)$ training pairs without numerical discretization. Contribution/Results: This work introduces the first “solution-to-source” inverse generation paradigm, eliminating dependence on numerical solvers while preserving mathematical rigor and computational scalability. Experiments demonstrate that neural operators trained exclusively on synthetic data achieve generalization performance on multiple PDE benchmarks comparable to—or even exceeding—that of models trained on solver-generated data.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Web Mining and Content Analysis: Web data generation and simulation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advances in the literature show promising potential of deep learning methods, particularly neural operators, in obtaining numerical solutions to partial differential equations (PDEs) beyond the reach of current numerical solvers. However, existing data-driven approaches often rely on training data produced by numerical PDE solvers (e.g., finite difference or finite element methods). We introduce a"backward"data generation method that avoids solving the PDE numerically: by randomly sampling candidate solutions $u_j$ from the appropriate solution space (e.g., $H_0^1(Omega)$), we compute the corresponding right-hand side $f_j$ directly from the equation by differentiation. This produces training pairs ${(f_j, u_j)}$ by computing derivatives rather than solving a PDE numerically for each data point, enabling fast, large-scale data generation consisting of exact solutions. Experiments indicate that models trained on this synthetic data generalize well when tested on data produced by standard solvers. While the idea is simple, we hope this method will expand the potential of neural PDE solvers that do not rely on classical numerical solvers to generate their data.

Problem

• Generating synthetic data for neural operators without numerical PDE solvers
• Creating training pairs by computing derivatives instead of solving PDEs
• Enabling fast large-scale data generation with exact solutions

Innovation

• Generates synthetic data without numerical PDE solving
• Samples solutions and computes right-hand sides directly
• Enables fast large-scale exact solution data generation