SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models

2026-05-18

AI Summary

Existing compression methods for physics-informed foundation models often overlook the functional nature of physical data and its sensitivity to high-order partial derivatives, leading to significant degradation in both accuracy and physical fidelity. This work proposes a sensitivity-aware singular value decomposition (SVD) compression framework that explicitly models layer-wise sensitivity in the output function space through a loss-aware mechanism and incorporates fidelity constraints to preserve high-order derivative structures. By introducing function-space sensitivity modeling into physics-informed model compression for the first time, the proposed approach substantially outperforms existing techniques across multiple benchmarks: it achieves higher compression ratios while maintaining or even improving model accuracy, with up to an order-of-magnitude gain in compression efficiency in certain scenarios, thereby paving a new path toward deployable and sustainable scientific foundation models.

Abstract

We propose a new method for compressing physics foundation models (PFMs) which is a new trend in AI for Science. While model compression is essential for reducing memory use and accelerating inference in large foundation models, it remains under-explored for PFMs, where preserving physical fidelity is crucial. The challenge lies in the functional nature of physics data, where partial derivatives encode spatiotemporal dynamics and exhibit high sensitivity to compression. Conventional compression methods ignore this structure, often causing severe performance degradation or failure. To address this, we introduce a sensitivity-aware fidelity-enforcing compression framework that explicitly models loss-aware layer sensitivity in the output function space during compression. This provides a new route to compressing scientific foundation models while preserving accuracy and physical fidelity. Experiments show substantial gains over existing methods across multiple models and datasets, achieving significantly higher compression ratios while maintaining accuracy, in some cases by orders of magnitude. More broadly, the work potentially leads to a new subfield of efficient, deployable, and sustainable scientific foundation models in AI for Science.

Problem

• physics foundation models
• model compression
• physical fidelity
• sensitivity
• functional data

Innovation

• sensitivity-aware compression
• fidelity-enforcing
• physics foundation models
• function space sensitivity
• SVD-based compression