MoCoLSK: Modality Conditioned High-Resolution Downscaling for Land Surface Temperature

2024-09-30

AI Summary

To address the long-standing spatial–temporal resolution trade-off in land surface temperature (LST) retrieval from satellite remote sensing, this paper proposes the Modality-Conditioned Large Selection Kernel network (MoCoLSK), which enhances LST downscaling accuracy via dynamic multimodal feature fusion and adaptive receptive field modulation. Key innovations include a modality-conditioned projection mechanism and a dynamic selection kernel architecture. We further introduce GrokLST—the first open-source deep learning ecosystem for LST—comprising a high-quality benchmark dataset, a unified evaluation framework, and a toolkit of 40+ algorithms. Extensive experiments across diverse real-world multisource remote sensing scenarios demonstrate that MoCoLSK reduces LST prediction RMSE by 12.7% on average over state-of-the-art methods. All code, datasets, and tools are publicly released and have been widely adopted by the remote sensing community.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Remote Sensing / Geospatial AI
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Land Surface Temperature (LST) is a critical parameter for environmental studies, but directly obtaining high spatial resolution LST data remains challenging due to the spatio-temporal trade-off in satellite remote sensing. Guided LST downscaling has emerged as an alternative solution to overcome these limitations, but current methods often neglect spatial non-stationarity, and there is a lack of an open-source ecosystem for deep learning methods. In this paper, we propose the Modality-Conditional Large Selective Kernel (MoCoLSK) Network, a novel architecture that dynamically fuses multi-modal data through modality-conditioned projections. MoCoLSK achieves a confluence of dynamic receptive field adjustment and multi-modal feature fusion, leading to enhanced LST prediction accuracy. Furthermore, we establish the GrokLST project, a comprehensive open-source ecosystem featuring the GrokLST dataset, a high-resolution benchmark, and the GrokLST toolkit, an open-source PyTorch-based toolkit encapsulating MoCoLSK alongside 40+ state-of-the-art approaches. Extensive experimental results validate MoCoLSK's effectiveness in capturing complex dependencies and subtle variations within multispectral data, outperforming existing methods in LST downscaling. Our code, dataset, and toolkit are available at https://github.com/GrokCV/GrokLST.

Problem

• Overcomes spatio-temporal trade-off in high-resolution LST data acquisition.
• Addresses spatial non-stationarity in guided LST downscaling methods.
• Provides an open-source ecosystem for deep learning in LST downscaling.

Innovation

• Dynamic receptive field adjustment for LST downscaling
• Modality-conditioned multi-modal data fusion
• Open-source ecosystem with dataset and toolkit