MotifGen: Spatiotemporal interpolation of misaligned satellite images via multi-source generative modeling, in an application to tropical cyclones

2026-06-23

AI Summary

Continuous monitoring of tropical cyclones is hindered by the long revisit intervals of microwave satellite imagery, sensor heterogeneity, irregular spatiotemporal sampling, and geographic misalignment across observations. To address these challenges, this work proposes a multi-source generative model that leverages self-supervised learning to perform spatiotemporal interpolation of heterogeneous microwave and infrared satellite images, enabling the first joint modeling of unaligned, non-uniformly sampled data. The approach incorporates a random source masking and reconstruction strategy to enhance generalization. Experiments demonstrate that the model significantly outperforms supervised baselines, as measured by reduced Continuous Ranked Probability Score (CRPS), with performance further improved through infrared data fusion. The generated outputs exhibit power spectra closely matching real observations, and their ensemble mean rivals that of deterministic forecast models.

Abstract

Microwave satellite imagery plays a crucial role in monitoring tropical cyclone precipitation and intensity worldwide, but suffers from long revisit times, potentially missing rapid storm evolution phases. While this raises the need for an interpolation method, it is made challenging by the high level of heterogeneity of microwave data coming from different instruments. In this work, we introduce the first generative model that can be applied to multiple geospatial sources that change across samples, occur at irregular time intervals, are misaligned geographically, and come from instruments with varying characteristics. We apply this model to the case of spatio-temporal interpolation of tropical cyclone microwave images from other microwave and infrared instruments. We train using a self-supervised task in which a random source is masked and reconstructed, and show that it leads to a significant decrease in Continuous Ranked Probability Score over supervised training. We show a further improvement by combining infrared and microwave data compared to microwave only. Using these improvements, the generative model produces an ensemble mean on par with that of a deterministic model, while generating a power spectrum significantly closer to that of true observations. To the best of our knowledge, this is the first generative model that interpolates microwave images of cyclones by combining multiple microwave instruments and infrared observations at irregular time intervals.

Problem

• spatiotemporal interpolation
• misaligned satellite images
• tropical cyclones
• multi-source data
• microwave imagery

Innovation

• generative modeling
• spatiotemporal interpolation
• multi-source satellite data
• microwave imagery
• tropical cyclones