CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training

2026-07-08

AI Summary

Existing remote sensing approaches struggle to consistently estimate urban carbon emissions across cities due to data heterogeneity and the absence of fine-grained semantic–temporal context. This work proposes CarbonCLIP, a task-oriented multimodal distillation framework that integrates street-view-derived textual semantic priors—generated by a large vision–language model—with monthly temporal encodings into satellite representations through a dual-branch contrastive learning scheme. The method enables a transition from multimodal pretraining to unimodal inference using only satellite imagery, thereby overcoming the limitations of static visual features. Experiments in Beijing and Singapore demonstrate substantial improvements over baseline methods, confirming the framework’s effectiveness in enhancing both the accuracy of urban carbon emission prediction and its cross-city generalizability.

Abstract

Accurately estimating urban carbon emissions is critical for sustainable urban planning, yet many existing approaches remain difficult to apply consistently across cities due to data-source heterogeneity and the lack of fine-grained semantic-temporal context in remote sensing data. We propose CarbonCLIP, a task-oriented multimodal distillation framework that improves satellite-based carbon emission prediction by transferring contextual knowledge into a unified satellite representation through dual-branch contrastive learning. Unlike conventional methods that rely on static visual features, CarbonCLIP explicitly bridges the gap between top-down satellite views and ground-level human activities. Specifically, the spatial branch uses fine-grained textual descriptions automatically generated from street-view images by Large Multimodal Models (LMMs) to provide semantic priors reflecting building functions, infrastructure, and urban activities, while the temporal branch employs a month encoder to encode temporal priors associated with monthly emission variation. CarbonCLIP requires multimodal data only during the pretraining phase; during inference, it relies solely on satellite imagery, thereby supporting scalable deployment when ground-level data are unavailable at inference. Experiments on Beijing and Singapore demonstrate that CarbonCLIP outperforms baselines in both study cities. The results validate that our method effectively transfers multimodal knowledge into satellite representations, offering a robust solution for satellite-based urban carbon modeling.

Problem

• urban carbon emissions
• satellite imagery
• semantic context
• temporal context
• data heterogeneity

Innovation

• multimodal distillation
• contrastive learning
• satellite carbon prediction
• street-view semantics
• temporal context