MASH: A Multiplatform and Multimodal Annotated Dataset for Societal Impact of Hurricane

2025-09-28

AI Summary

Existing hurricane social impact datasets suffer from three key limitations: temporal latency, platform homogeneity, and modality fragmentation (i.e., disjoint text and image annotations). To address these gaps, we introduce MASH—the first large-scale, multi-platform, multimodal dataset for hurricane-related social impact analysis, comprising 98,662 samples from Reddit, X (formerly Twitter), TikTok, and YouTube. MASH pioneers cross-platform, multimodal, and multidimensional joint annotation, integrating textual and visual content across three critical dimensions: humanitarian response, bias detection, and information integrity. Leveraging human-in-the-loop multimodal analysis techniques, it supports fine-grained classification tasks. By unifying heterogeneous social media modalities and platforms, MASH fills a critical data void in multimodal natural disaster impact assessment. It serves as a foundational resource for modeling disaster severity, analyzing public sentiment, informing policy interventions, and advancing research on algorithmic fairness in crisis contexts.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Natural disasters cause multidimensional threats to human societies, with hurricanes exemplifying one of the most disruptive events that not only caused severe physical damage but also sparked widespread discussion on social media platforms. Existing datasets for studying societal impacts of hurricanes often focus on outdated hurricanes and are limited to a single social media platform, failing to capture the broader societal impact in today's diverse social media environment. Moreover, existing datasets annotate visual and textual content of the post separately, failing to account for the multimodal nature of social media posts. To address these gaps, we present a multiplatform and Multimodal Annotated Dataset for Societal Impact of Hurricane (MASH) that includes 98,662 relevant social media data posts from Reddit, X, TikTok, and YouTube. In addition, all relevant social media data posts are annotated in a multimodal approach that considers both textual and visual content on three dimensions: humanitarian classes, bias classes, and information integrity classes. To our best knowledge, MASH is the first large-scale, multi-platform, multimodal, and multi-dimensionally annotated hurricane dataset. We envision that MASH can contribute to the study of hurricanes' impact on society, such as disaster severity classification, public sentiment analysis, disaster policy making, and bias identification.

Problem

• Addresses lack of multiplatform hurricane impact datasets
• Solves unimodal annotation of social media content
• Provides multidimensional labeling for disaster analysis

Innovation

• Multiplatform hurricane dataset from four social media sources
• Multimodal annotation combining text and visual content
• Three-dimensional labeling for humanitarian, bias, and information integrity