Physics-Guided Foundation Model for Scientific Discovery: An Application to Aquatic Science

2025-02-10

AI Summary

Existing physics-guided machine learning (PGML) methods primarily target isolated, simple tasks and struggle to model complex systems involving coupled physical processes. Method: We propose Physics-Guided Foundation Models (PGFMs), the first framework to embed physical constraints into a foundation model architecture. PGFMs achieve cross-process generalization via multi-task-guided adaptive feature interaction and strictly enforce mass/energy conservation throughout the pretraining–fine-tuning pipeline. The approach integrates physics-simulation-driven self-supervised pretraining, multi-task learning, physics-constrained fine-tuning, and a synergistic neural–mechanistic modeling architecture. Contribution/Results: Evaluated on real-world lake data, PGFMs reduce mean prediction errors for water temperature and dissolved oxygen by 32% compared to conventional PGML methods, significantly improving accuracy in modeling coupled biogeochemical–hydrodynamic processes. Moreover, the framework demonstrates strong transferability to other physics-driven scientific domains.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Physics-guided machine learning (PGML) has become a prevalent approach in studying scientific systems due to its ability to integrate scientific theories for enhancing machine learning (ML) models. However, most PGML approaches are tailored to isolated and relatively simple tasks, which limits their applicability to complex systems involving multiple interacting processes and numerous influencing features. In this paper, we propose a extit{ extbf{P}hysics- extbf{G}uided extbf{F}oundation extbf{M}odel ( extbf{PGFM})} that combines pre-trained ML models and physics-based models and leverages their complementary strengths to improve the modeling of multiple coupled processes. To effectively conduct pre-training, we construct a simulated environmental system that encompasses a wide range of influencing features and various simulated variables generated by physics-based models. The model is pre-trained in this system to adaptively select important feature interactions guided by multi-task objectives. We then fine-tune the model for each specific task using true observations, while maintaining consistency with established physical theories, such as the principles of mass and energy conservation. We demonstrate the effectiveness of this methodology in modeling water temperature and dissolved oxygen dynamics in real-world lakes. The proposed PGFM is also broadly applicable to a range of scientific fields where physics-based models are being used.

Problem

• Physics-Guided Foundation Model
• Multiple coupled processes modeling
• Water temperature and dissolved oxygen dynamics

Innovation

• Physics-Guided Foundation Model
• Pre-trained ML and physics models
• Multi-task objectives guide features