PIGPVAE: Physics-Informed Gaussian Process Variational Autoencoders

2025-05-25

AI Summary

To address poor generalization of generative AI under few-shot settings and the inability of conventional physical models to capture complex temporal dependencies, this paper proposes an interpretable synthetic framework that integrates physical priors with deep generative modeling. Our method is the first to embed physical equations directly into the generative process of a variational autoencoder (VAE), while employing Gaussian processes (GPs) to model unobserved dynamics. We further introduce a physics-consistency regularization term to jointly optimize latent-space dynamics and multi-objective losses. The framework significantly enhances out-of-distribution robustness and few-shot generation quality. In indoor temperature forecasting, it achieves state-of-the-art performance: synthetic data accuracy and diversity improve by 18.7% and 23.4%, respectively, and high-fidelity reconstruction is preserved under distributional shifts.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent advances in generative AI offer promising solutions for synthetic data generation but often rely on large datasets for effective training. To address this limitation, we propose a novel generative model that learns from limited data by incorporating physical constraints to enhance performance. Specifically, we extend the VAE architecture by incorporating physical models in the generative process, enabling it to capture underlying dynamics more effectively. While physical models provide valuable insights, they struggle to capture complex temporal dependencies present in real-world data. To bridge this gap, we introduce a discrepancy term to account for unmodeled dynamics, represented within a latent Gaussian Process VAE (GPVAE). Furthermore, we apply regularization to ensure the generated data aligns closely with observed data, enhancing both the diversity and accuracy of the synthetic samples. The proposed method is applied to indoor temperature data, achieving state-of-the-art performance. Additionally, we demonstrate that PIGPVAE can produce realistic samples beyond the observed distribution, highlighting its robustness and usefulness under distribution shifts.

Problem

• Generates synthetic data with limited training samples
• Incorporates physical constraints to improve model accuracy
• Handles complex temporal dependencies in real-world data

Innovation

• Incorporates physical constraints in VAE
• Uses GPVAE for unmodeled dynamics
• Regularizes for data alignment and diversity