Post-hoc Study of Climate Microtargeting on Social Media Ads with LLMs: Thematic Insights and Fairness Evaluation

2024-10-07

arXiv.org

AI Summary

This study investigates demographic targeting patterns and fairness risks associated with microtargeting strategies in climate-related social media advertising. We propose the first post-hoc explainability framework leveraging large language models (LLMs), integrating prompt engineering, text classification, and fairness auditing to identify gender- and age-based targeting and attribute thematic content in Facebook climate ads. Our model achieves 88.55% accuracy in predicting targeted demographics. Results reveal that youth are significantly associated with activist narratives, while women are disproportionately exposed to caregiving-themed content. Systematic classification biases are identified against older adults and men. The key contribution lies in pioneering the joint application of LLMs for explainable and fairness-aware evaluation of climate communication microtargeting—establishing a methodological foundation and empirical evidence for equitable environmental information dissemination in digital environments.

Technology Category

• Machine Learning: Ethics, Bias, and Fairness
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Application Domains: Humanities & Computational Social Science

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Climate change communication on social media increasingly employs microtargeting strategies to effectively reach and influence specific demographic groups. This study presents a post-hoc analysis of microtargeting practices within climate campaigns by leveraging large language models (LLMs) to examine Facebook advertisements. Our analysis focuses on two key aspects: demographic targeting and fairness. We evaluate the ability of LLMs to accurately predict the intended demographic targets, such as gender and age group, achieving an overall accuracy of 88.55%. Furthermore, we instruct the LLMs to generate explanations for their classifications, providing transparent reasoning behind each decision. These explanations reveal the specific thematic elements used to engage different demographic segments, highlighting distinct strategies tailored to various audiences. Our findings show that young adults are primarily targeted through messages emphasizing activism and environmental consciousness, while women are engaged through themes related to caregiving roles and social advocacy. In addition to evaluating the effectiveness of LLMs in detecting microtargeted messaging, we conduct a comprehensive fairness analysis to identify potential biases in model predictions. Our findings indicate that while LLMs perform well overall, certain biases exist, particularly in the classification of senior citizens and male audiences. By showcasing the efficacy of LLMs in dissecting and explaining targeted communication strategies and by highlighting fairness concerns, this study provides a valuable framework for future research aimed at enhancing transparency, accountability, and inclusivity in social media-driven climate campaigns.

Problem

• Analyzing climate microtargeting on social media using LLMs
• Evaluating fairness and accuracy of LLM demographic predictions
• Identifying thematic strategies for engaging different audience segments

Innovation

• LLMs analyze Facebook ads for microtargeting insights
• LLMs predict demographics with 88.55% accuracy
• Fairness analysis reveals biases in LLM classifications