Interpreting "Interpretability" and Explaining "Explainability" in Machine Learning in Physics

2026-06-24

AI Summary

This study clarifies the conceptual confusion in physics-oriented machine learning between “interpretability”—referring to model transparency—and “explainability,” which denotes the capacity to map onto domain knowledge. It delineates the boundaries of these two notions and examines their trade-offs in terms of expressive power and adaptability. Through conceptual analysis and the construction of a unifying framework, complemented by a systematic review of both intrinsic and post-hoc explanation methods, the work advocates for integrating interpretability and explainability into scientific modeling paradigms. Crucially, it underscores the central role of task formulation and intervention design in model development. By establishing a clear conceptual foundation and methodological guidance, this research advances the principled integration of machine learning models with scientific reasoning in physics.

Abstract

We review the concepts of interpretability and explainability as they apply to machine learning in physics. We define interpretability as concerning the structural transparency of a model (the ability to understand or approximate its inner workings) and explainability as concerning the scientific content of a model (the ability to map it onto domain knowledge). We discuss the trade-offs each entails (interpretability vs. expressivity; explainability vs. adaptability), the contexts in which each is needed, and the intrinsic and post-hoc tools available for achieving them. Throughout, we emphasize that machine-learned models are subject to the same scientific questions as classical models, differing only in scale, and that interpretability and explainability are best understood as deliberate modeling choices rather than inherent properties. We also emphasize the importance of task specification and intervention plans as a core aspect of model design.

Problem

• interpretability
• explainability
• machine learning
• physics
• scientific modeling

Innovation

• interpretability
• explainability
• machine learning in physics
• model transparency
• scientific modeling