Towards a Physics Foundation Model

2025-09-17

AI Summary

Existing physics-informed machine learning models exhibit poor generalizability, requiring system-specific retraining and thus impeding universal scientific simulation. Method: We propose the General-purpose Physics Transformer (GPhyT), the first foundational model enabling zero-shot cross-physics generalization—deployable on unseen systems without fine-tuning. Built upon the Transformer architecture, GPhyT is pre-trained on 1.8 TB of heterogeneous, multi-physics simulation data, integrating context-aware learning and sequence modeling to implicitly encode conservation laws and dynamical principles. Contribution/Results: Experiments demonstrate that GPhyT achieves up to 29× higher prediction accuracy than domain-specific models across diverse physical domains. It enables stable 50-step autoregressive rollout, accurately captures long-term evolution, and models complex multi-phase dynamics. GPhyT establishes a new simulation paradigm for computational science: transferable, high-fidelity, and free of retraining.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Low Level & Physics-based Vision
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Foundation models have revolutionized natural language processing through a ``train once, deploy anywhere'' paradigm, where a single pre-trained model adapts to countless downstream tasks without retraining. Access to a Physics Foundation Model (PFM) would be transformative -- democratizing access to high-fidelity simulations, accelerating scientific discovery, and eliminating the need for specialized solver development. Yet current physics-aware machine learning approaches remain fundamentally limited to single, narrow domains and require retraining for each new system. We present the General Physics Transformer (GPhyT), trained on 1.8 TB of diverse simulation data, that demonstrates foundation model capabilities are achievable for physics. Our key insight is that transformers can learn to infer governing dynamics from context, enabling a single model to simulate fluid-solid interactions, shock waves, thermal convection, and multi-phase dynamics without being told the underlying equations. GPhyT achieves three critical breakthroughs: (1) superior performance across multiple physics domains, outperforming specialized architectures by up to 29x, (2) zero-shot generalization to entirely unseen physical systems through in-context learning, and (3) stable long-term predictions through 50-timestep rollouts. By establishing that a single model can learn generalizable physical principles from data alone, this work opens the path toward a universal PFM that could transform computational science and engineering.

Problem

• Developing a single model for multiple physics domains
• Achieving zero-shot generalization to unseen physical systems
• Enabling stable long-term predictions without retraining

Innovation

• Transformer trained on diverse simulation data
• Learns governing dynamics from context alone
• Zero-shot generalization to unseen physical systems