Multi-Modal Framing Analysis of News

2025-03-26

AI Summary

Existing political communication framing analyses predominantly rely on predefined textual frames, neglecting the synergistic editorial choices between text and images in news. Method: We propose the first theory-driven, multimodal, multi-label news framing analysis framework, tightly integrating framing theory with multimodal large language-vision models to enable implicit frame extraction and partisan orientation identification under cross-modal semantic alignment. Our approach leverages large-scale vision-language models, jointly optimizing contrastive learning and multi-label classification to construct a unified embedding space that disentangles and aligns frame semantics across modalities. Contribution/Results: On real-world news datasets, we achieve the first quantitative measurement of image-text framing consistency and issue-specific partisan orientation detection. This advances media bias analysis by significantly improving its comprehensiveness, interpretability, and fine-grained resolution.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Automated frame analysis of political communication is a popular task in computational social science that is used to study how authors select aspects of a topic to frame its reception. So far, such studies have been narrow, in that they use a fixed set of pre-defined frames and focus only on the text, ignoring the visual contexts in which those texts appear. Especially for framing in the news, this leaves out valuable information about editorial choices, which include not just the written article but also accompanying photographs. To overcome such limitations, we present a method for conducting multi-modal, multi-label framing analysis at scale using large (vision-)language models. Grounding our work in framing theory, we extract latent meaning embedded in images used to convey a certain point and contrast that to the text by comparing the respective frames used. We also identify highly partisan framing of topics with issue-specific frame analysis found in prior qualitative work. We demonstrate a method for doing scalable integrative framing analysis of both text and image in news, providing a more complete picture for understanding media bias.

Problem

• Analyzing multi-modal framing in news beyond text
• Identifying partisan framing using issue-specific analysis
• Integrating text and image for media bias understanding

Innovation

• Multi-modal framing analysis using large models
• Contrasting image and text frames for bias
• Scalable integrative analysis of news content