Towards Physics-Guided Foundation Models

2025-02-20

AI Summary

Traditional foundation models often violate physical laws during out-of-distribution (OOD) prediction, yielding physically infeasible outputs. To address this, we propose the Physics-Guided Foundation Model (PGFM) paradigm—the first systematic framework enabling universal integration of general-purpose physical priors (e.g., conservation laws, differential equation constraints) into large-model pretraining. Methodologically, PGFM incorporates differentiable physics equation embedding, multi-scale physics knowledge distillation, and science-aware pretraining objectives. It preserves downstream fine-tuning efficiency while substantially improving OOD generalization and physical consistency: across multiple scientific AI benchmarks, spurious prediction rates decrease by over 40%, without requiring task-specific physical modeling. This work establishes a scalable architectural foundation for building robust, trustworthy scientific foundation models grounded in fundamental physical principles.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Low Level & Physics-based Vision
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Traditional foundation models are pre-trained on broad datasets to reduce the training resources (e.g., time, energy, labeled samples) needed for fine-tuning a wide range of downstream tasks. However, traditional foundation models struggle with out-of-distribution prediction and can produce outputs that are unrealistic and physically infeasible. We propose the notation of physics-guided foundation models (PGFM), that is, foundation models integrated with broad or general domain (e.g., scientific) physical knowledge applicable to a wide range of downstream tasks.

Problem

• Enhance out-of-distribution prediction accuracy
• Ensure physically feasible model outputs
• Integrate physical knowledge into foundation models

Innovation

• Physics-guided foundation models
• Integrates scientific physical knowledge
• Enhances out-of-distribution prediction