Comparative Analysis of Transformer Models in Disaster Tweet Classification for Public Safety

2025-09-04

AI Summary

To address low classification accuracy of social media text—particularly disaster-related tweets—in public safety applications, this study systematically compares Transformer-based models (BERT, DistilBERT, RoBERTa, and DeBERTa) against traditional classifiers (logistic regression, naïve Bayes, and SVM) in handling informal and metaphorical language. Experimental results show that BERT achieves 91% accuracy, significantly outperforming traditional methods (all at 82%), thereby validating the superiority of attention-based, context-aware pre-trained language models in capturing deep semantic features. This work constitutes the first comprehensive empirical evaluation of both lightweight and enhanced Transformer architectures specifically for disaster tweet classification. It provides a reproducible technical framework and evidence-based insights for real-time, high-accuracy information filtering in emergency response systems.

Technology Category

• Natural Language Processing: Text Classification & Sentiment Analysis
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Twitter and other social media platforms have become vital sources of real time information during disasters and public safety emergencies. Automatically classifying disaster related tweets can help emergency services respond faster and more effectively. Traditional Machine Learning (ML) models such as Logistic Regression, Naive Bayes, and Support Vector Machines have been widely used for this task, but they often fail to understand the context or deeper meaning of words, especially when the language is informal, metaphorical, or ambiguous. We posit that, in this context, transformer based models can perform better than traditional ML models. In this paper, we evaluate the effectiveness of transformer based models, including BERT, DistilBERT, RoBERTa, and DeBERTa, for classifying disaster related tweets. These models are compared with traditional ML approaches to highlight the performance gap. Experimental results show that BERT achieved the highest accuracy (91%), significantly outperforming traditional models like Logistic Regression and Naive Bayes (both at 82%). The use of contextual embeddings and attention mechanisms allows transformer models to better understand subtle language in tweets, where traditional ML models fall short. This research demonstrates that transformer architectures are far more suitable for public safety applications, offering improved accuracy, deeper language understanding, and better generalization across real world social media text.

Problem

• Classifying disaster-related tweets for emergency response
• Comparing transformer models with traditional ML approaches
• Improving language understanding in informal social media text

Innovation

• Transformer models for disaster tweet classification
• BERT achieved highest accuracy outperforming traditional models
• Contextual embeddings and attention mechanisms improve understanding