MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing

2025-07-22

AI Summary

To address three critical bottlenecks in natural disaster monitoring—poor model generalizability, low automation, and scarcity of fine-grained multimodal annotations—this paper introduces the first large-scale multimodal remote sensing dataset covering over 10,000 FEMA-declared disaster events. The dataset integrates high-temporal-resolution satellite imagery, news articles, geospatial coordinates, and structured question-answer pairs. We pioneer the application of multimodal large language models (MLLMs) to disaster response, proposing a novel framework for cross-modal alignment and temporal semantic reasoning to enable unified modeling across disaster types and natural-language-driven evolutionary analysis. Our method achieves significant performance gains on disaster detection, localization, and attribution tasks. It establishes a new benchmark for machine learning–augmented emergency response. Both the codebase and dataset are fully open-sourced.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Natural disasters cause devastating damage to communities and infrastructure every year. Effective disaster response is hampered by the difficulty of accessing affected areas during and after events. Remote sensing has allowed us to monitor natural disasters in a remote way. More recently there have been advances in computer vision and deep learning that help automate satellite imagery analysis, However, they remain limited by their narrow focus on specific disaster types, reliance on manual expert interpretation, and lack of datasets with sufficient temporal granularity or natural language annotations for tracking disaster progression. We present MONITRS, a novel multimodal dataset of more than 10,000 FEMA disaster events with temporal satellite imagery and natural language annotations from news articles, accompanied by geotagged locations, and question-answer pairs. We demonstrate that fine-tuning existing MLLMs on our dataset yields significant performance improvements for disaster monitoring tasks, establishing a new benchmark for machine learning-assisted disaster response systems. Code can be found at: https://github.com/ShreelekhaR/MONITRS

Problem

• Lack of comprehensive datasets for tracking diverse disaster types
• Reliance on manual expert interpretation limits scalability
• Insufficient temporal granularity in existing disaster monitoring systems

Innovation

• Multimodal dataset with satellite and news data
• Fine-tuning MLLMs for disaster monitoring tasks
• Geotagged locations and question-answer pairs included