PropaInsight: Toward Deeper Understanding of Propaganda in Terms of Techniques, Appeals, and Intent

2024-09-19

arXiv.org

AI Summary

Prior work primarily focuses on identifying propaganda techniques, failing to model their underlying motivations and cross-contextual effects. Method: We propose PropaInsight, the first framework integrating technical execution, emotional appeals, and deep intent into a unified three-dimensional analytical paradigm—overcoming limitations of unidimensional technique detection. We introduce Propagaze, the first high-quality, multidimensionally annotated dataset grounded in sociological theory, human annotation, and controllable synthesis—addressing annotation scarcity and cross-domain generalization challenges. Leveraging PropaInsight, we design a controllable synthesis pipeline and fine-tune Llama-7B-Chat for end-to-end multi-task joint parsing. Results: Our approach achieves a 203.4% improvement in technique localization (IoU), outperforms 1-shot GPT-4-Turbo by 66.2 points in appeal analysis (BERTScore), and significantly enhances robustness under low-resource and cross-domain settings.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Application Domains: Humanities & Computational Social Science
• Humans and AI: Interaction Techniques and Devices

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Propaganda plays a critical role in shaping public opinion and fueling disinformation. While existing research primarily focuses on identifying propaganda techniques, it lacks the ability to capture the broader motives and the impacts of such content. To address these challenges, we introduce propainsight, a conceptual framework grounded in foundational social science research, which systematically dissects propaganda into techniques, arousal appeals, and underlying intent. propainsight offers a more granular understanding of how propaganda operates across different contexts. Additionally, we present propagaze, a novel dataset that combines human-annotated data with high-quality synthetic data generated through a meticulously designed pipeline. Our experiments show that off-the-shelf LLMs struggle with propaganda analysis, but training with propagaze significantly improves performance. Fine-tuned Llama-7B-Chat achieves 203.4% higher text span IoU in technique identification and 66.2% higher BertScore in appeal analysis compared to 1-shot GPT-4-Turbo. Moreover, propagaze complements limited human-annotated data in data-sparse and cross-domain scenarios, showing its potential for comprehensive and generalizable propaganda analysis.

Problem

• Analyzes propaganda techniques and motives
• Introduces Propainsight framework for deeper understanding
• Uses Propagaze dataset to enhance analysis accuracy

Innovation

• Conceptual framework for propaganda analysis
• Novel dataset combining human and synthetic data
• Fine-tuned LLM for enhanced performance