GateNLP at SemEval-2025 Task 10: Hierarchical Three-Step Prompting for Multilingual Narrative Classification

2025-05-28

AI Summary

To address the challenges of multilingual news proliferation and escalating misinformation, this paper proposes a two-level news narrative classification method—comprising primary and sub-narratives—for the domains of the Ukraine–Russia war and climate change. We introduce H3Prompt, an innovative three-tier hierarchical prompting framework (domain → primary narrative → sub-narrative) that enables zero-shot, cross-lingual inference via large language models for domain identification, primary narrative retrieval, and sub-narrative generation. The framework achieves fine-grained modeling, strong interpretability, and robust multilingual generalization. Evaluated on the SemEval-2025 Task 10 English test set, our approach ranks first among 28 participating teams, achieving state-of-the-art performance in both sub-narrative classification accuracy and label consistency.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Misinformation & Fake News

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

The proliferation of online news and the increasing spread of misinformation necessitate robust methods for automatic data analysis. Narrative classification is emerging as a important task, since identifying what is being said online is critical for fact-checkers, policy markers and other professionals working on information studies. This paper presents our approach to SemEval 2025 Task 10 Subtask 2, which aims to classify news articles into a pre-defined two-level taxonomy of main narratives and sub-narratives across multiple languages. We propose Hierarchical Three-Step Prompting (H3Prompt) for multilingual narrative classification. Our methodology follows a three-step Large Language Model (LLM) prompting strategy, where the model first categorises an article into one of two domains (Ukraine-Russia War or Climate Change), then identifies the most relevant main narratives, and finally assigns sub-narratives. Our approach secured the top position on the English test set among 28 competing teams worldwide. The code is available at https://github.com/GateNLP/H3Prompt.

Problem

• Classify news articles into narratives and sub-narratives
• Address multilingual narrative classification challenges
• Detect misinformation via hierarchical narrative taxonomy

Innovation

• Hierarchical Three-Step Prompting (H3Prompt) for classification
• Multilingual narrative classification using LLM prompting
• Three-step strategy: domain, main, and sub-narrative identification