Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making

2026-04-14

AI Summary

This study addresses the challenge that the black-box nature of surrogate models in complex system simulation impedes understanding of input–response relationships, while existing explainable artificial intelligence (XAI) methods struggle to satisfy engineering constraints such as high fidelity, dynamic adaptability, and reliability. To bridge this gap, the work systematically integrates XAI techniques across the entire surrogate modeling pipeline, proposing a structured framework that synergistically combines XAI with surrogate modeling through both equation-driven and agent-based simulation case studies. By incorporating feature importance, local and global interpretability methods, and interaction effect analysis—while aligning with practical engineering requirements—the research clarifies the contextual applicability of XAI approaches and advocates for embedding interpretability throughout simulation-driven decision-making, thereby shifting the role of simulation from mere computational acceleration toward generating actionable insights.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Search and Optimization: Sampling/Simulation-based Search
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The simulation of complex systems increasingly relies on sophisticated but fundamentally opaque computational black-box simulators. Surrogate models play a central role in reducing the computational cost of complex systems simulations across a wide range of scientific and engineering domains. Notwithstanding, they inevitably inherit and often exacerbate this black-box nature, obscuring how input variables drive physical responses. Conversely, Explainable Artificial Intelligence (XAI) offers powerful tools to unpack these models. Yet, XAI methods struggle with engineering-specific constraints, such as highly correlated inputs, dynamical systems, and rigorous reliability requirements. Consequently, surrogate modeling and XAI have largely evolved as distinct fields of research, despite their strong complementarity. To reconnect these approaches, this state-of-the-art survey provides a structured perspective that maps existing XAI techniques onto the various stages of surrogate modeling workflows for design and exploration. To ground this synthesis, we draw upon illustrative applications across both equation-based simulations and agent-based modeling. We survey a broad spectrum of techniques, highlighting their strengths for revealing interactions and supporting human comprehension. Finally, we identify pressing open challenges, including the explainability of dynamical systems and the handling of mixed-variable systems, and propose a research agenda to make explainability a core, embedded element of simulation-driven workflows from model construction through decision-making. By transforming opaque emulators into explainable tools, this agenda empowers practitioners to move beyond accelerating simulations to extracting actionable insights from complex system behaviors.

Problem

• surrogate modeling
• explainable AI
• black-box models
• dynamical systems
• simulation interpretability

Innovation

• Surrogate Modeling
• Explainable AI (XAI)
• Interpretable Simulation
• Dynamical Systems
• Decision-Making