Late Fusion Neural Operators for Extrapolation Across Parameter Space in Partial Differential Equations

2026-04-17

AI Summary

This work addresses the limited generalization of neural operators when extrapolating beyond the parameter space of partial differential equations (PDEs), a challenge arising from the tight coupling between parameters and system states. To mitigate this, the authors propose a late-fusion architecture that explicitly decouples the learning of state dynamics from the incorporation of parameter effects. Specifically, a neural operator—such as the Fourier Neural Operator (FNO)—is employed to learn latent state representations, while parameter information is introduced in a structured manner via sparse regression. Evaluated on four PDE benchmark tasks, the proposed method substantially outperforms both standard FNO and CAPE-FNO, achieving average RMSE reductions of 72.9% within the training distribution and 71.8% outside it, thereby significantly enhancing predictive accuracy and extrapolation capability both in-domain and out-of-domain.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Developing neural operators that accurately predict the behavior of systems governed by partial differential equations (PDEs) across unseen parameter regimes is crucial for robust generalization in scientific and engineering applications. In practical applications, variations in physical parameters induce distribution shifts between training and prediction regimes, making extrapolation a central challenge. As a result, the way parameters are incorporated into neural operator models plays a key role in their ability to generalize, particularly when state and parameter representations are entangled. In this work, we introduce the Late Fusion Neural Operator, an architecture that disentangles learning state dynamics from parameter effects, improving predictive performance both within and beyond the training distribution. Our approach combines neural operators for learning latent state representations with sparse regression to incorporate parameter information in a structured manner. Across four benchmark PDEs including advection, Burgers, and both 1D and 2D reaction-diffusion equations, the proposed method consistently outperforms Fourier Neural Operator and CAPE-FNO. Late Fusion Neural Operators achieve consistently the best performance in all experiments, with an average RMSE reduction of 72.9% in-domain and 71.8% out-domain compared to the second-best method. These results demonstrate strong generalization across both in-domain and out-domain parameter regimes.

Problem

• neural operators
• extrapolation
• parameter space
• partial differential equations
• distribution shift

Innovation

• Late Fusion
• Neural Operators
• Parameter Extrapolation
• Disentangled Representation
• PDE Surrogate Modeling