Transferring Physical Priors into Remote Sensing Segmentation via Large Language Models

2026-03-28

AI Summary

This study addresses two key challenges in semantic segmentation of remote sensing imagery: the difficulty of effectively fusing optical images with misaligned physical variables (e.g., DEM, SAR, NDVI), and the prohibitive cost of retraining foundation models when integrating new sensors. To overcome these issues, the authors propose PriorSeg, a novel framework that leverages a large language model to construct a Physics-Centric Knowledge Graph (PCKG), which enables the generation of a heterogeneous yet aligned dataset, Phy-Sky-SA. By incorporating physical priors through vision–physics joint training and a residual refinement mechanism guided by physics-consistency loss, PriorSeg enhances segmentation accuracy and physical plausibility without requiring retraining of the underlying foundation model. Ablation studies confirm the effectiveness of each proposed component.

Abstract

Semantic segmentation of remote sensing imagery is fundamental to Earth observation. Achieving accurate results requires integrating not only optical images but also physical variables such as the Digital Elevation Model (DEM), Synthetic Aperture Radar (SAR) and Normalized Difference Vegetation Index (NDVI). Recent foundation models (FMs) leverage pre-training to exploit these variables but still depend on spatially aligned data and costly retraining when involving new sensors. To overcome these limitations, we introduce a novel paradigm for integrating domain-specific physical priors into segmentation models. We first construct a Physical-Centric Knowledge Graph (PCKG) by prompting large language models to extract physical priors from 1,763 vocabularies, and use it to build a heterogeneous, spatial-aligned dataset, Phy-Sky-SA. Building on this foundation, we develop PriorSeg, a physics-aware residual refinement model trained with a joint visual-physical strategy that incorporates a novel physics-consistency loss. Experiments on heterogeneous settings demonstrate that PriorSeg improves segmentation accuracy and physical plausibility without retraining the FMs. Ablation studies verify the effectiveness of the Phy-Sky-SA dataset, the PCKG, and the physics-consistency loss.

Problem

• remote sensing segmentation
• physical priors
• foundation models
• semantic segmentation
• physics consistency

Innovation

• Physical Priors
• Large Language Models
• Knowledge Graph
• Physics-Consistency Loss
• Remote Sensing Segmentation