A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning

2026-05-25

AI Summary

This work addresses the lack of theoretical generalization guarantees for physics-informed machine learning under unbounded loss functions. It introduces, for the first time, a PAC-Bayesian framework to this domain, unifying data fidelity and physical equation residuals through a multi-task modeling perspective, thereby circumventing the looseness inherent in conventional union-bound approaches. By incorporating Sobolev- and Poincaré-type assumptions, a complexity measure based on input gradient norms, and a self-bounding-aware optimization algorithm, the study establishes a non-vacuous and significantly tighter high-probability generalization bound. This bound explicitly reveals a direct link between physical regularity and generalization performance and demonstrates its optimizability on standard PDE benchmarks, offering a rigorous statistical foundation for physics-informed models.

Abstract

Physics-informed machine learning (PIML) integrates mechanistic knowledge, typically in the form of partial differential equations (PDE), into data-driven models. Despite strong empirical performance, its statistical generalisation properties remain poorly understood, particularly in the regression setting with unbounded losses. Existing analyses rely on approximation or stability arguments and do not fully capture how physical structure influences generalisation from finite data. In this work, we develop a PAC-Bayesian framework for PIML that provides high-probability generalisation guarantees in the presence of unbounded losses. We adopt a multi-task perspective that jointly treats data fidelity, PDE residuals, initial and boundary conditions, avoiding the looseness induced by standard union-bound approaches. Our analysis leverages the structure of physics-informed objectives to derive novel bounds where the complexity scales with input-gradient norms of the losses, revealing a direct link between physical regularity and generalisation. We instantiate this framework under Sobolev and Poincaré-type assumptions, yielding two classes of bounds that trade off statistical complexity and smoothness in different regimes. Building on these results, we propose a self-bounding-aware learning algorithm that directly optimises tractable surrogates of the derived bounds, along with a practical procedure to estimate the associated constants in realistic settings. Empirical evaluations on standard PDE benchmarks demonstrate that our bounds are non-vacuous, significantly tighter than union-bound baselines, and can be effectively minimised during training. Overall, our results provide a principled statistical foundation for the generalisation of physics-informed models.

Problem

• generalisation
• physics-informed machine learning
• unbounded losses
• PAC-Bayesian
• statistical learning theory

Innovation

• PAC-Bayesian
• physics-informed machine learning
• generalisation bounds
• unbounded losses
• multi-task learning