SIMSHIFT: A Benchmark for Adapting Neural Surrogates to Distribution Shifts

2025-06-13

AI Summary

Neural surrogate models exhibit poor generalization to unseen PDE configurations—e.g., novel materials or geometries. To address this, we introduce SIMSHIFT, the first industrial-simulation-oriented benchmark for distribution shift, covering four real-world tasks: hot rolling, sheet metal forming, motor design, and heat sink optimization. We pioneer the systematic integration of domain adaptation (DA) into PDE neural surrogate modeling, proposing a novel adaptation paradigm: “parameterized descriptors + multi-source ground-truth simulations + zero target-domain ground truth.” We extend classical DA methods—including DANN, MMD, and CORAL—to state-of-the-art architectures (e.g., FourierNet, DeepONet), incorporating physics-informed parameter embedding and cross-configuration feature alignment. Experiments demonstrate that DA substantially improves prediction accuracy, reducing mean absolute error by 27–41% across tasks, while uncovering fundamental limitations of existing surrogates under strong geometric and physical distribution shifts. The benchmark dataset and implementation code are publicly released.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Neural surrogates for Partial Differential Equations (PDEs) often suffer significant performance degradation when evaluated on unseen problem configurations, such as novel material types or structural dimensions. Meanwhile, Domain Adaptation (DA) techniques have been widely used in vision and language processing to generalize from limited information about unseen configurations. In this work, we address this gap through two focused contributions. First, we introduce SIMSHIFT, a novel benchmark dataset and evaluation suite composed of four industrial simulation tasks: hot rolling, sheet metal forming, electric motor design and heatsink design. Second, we extend established domain adaptation methods to state of the art neural surrogates and systematically evaluate them. These approaches use parametric descriptions and ground truth simulations from multiple source configurations, together with only parametric descriptions from target configurations. The goal is to accurately predict target simulations without access to ground truth simulation data. Extensive experiments on SIMSHIFT highlight the challenges of out of distribution neural surrogate modeling, demonstrate the potential of DA in simulation, and reveal open problems in achieving robust neural surrogates under distribution shifts in industrially relevant scenarios. Our codebase is available at https://github.com/psetinek/simshift

Problem

• Address performance drop of neural PDE surrogates on unseen configurations
• Introduce SIMSHIFT benchmark for industrial simulation adaptation tasks
• Evaluate domain adaptation methods for robust neural surrogate predictions

Innovation

• Introduces SIMSHIFT benchmark for industrial simulations
• Extends domain adaptation to neural surrogates
• Uses parametric descriptions for target predictions