Polarity Detection of Sustainable Detection Goals in News Text

2025-09-24

AI Summary

This paper introduces SDG Polarity Detection, a novel task aimed at automatically identifying the directional impact—positive, neutral, or negative—of United Nations Sustainable Development Goals (SDGs) mentioned in news texts. To support this fine-grained semantic analysis, we construct SDG-POD, the first fully annotated benchmark dataset. We further pioneer the investigation of synthetic data augmentation for this task. Experiments systematically compare zero-shot inference and supervised fine-tuning across six state-of-the-art large language models. Results reveal limited zero-shot capability of current LLMs; however, fine-tuned QWQ-32B achieves significant gains over baselines across multiple SDG categories. Crucially, integrating real and synthetic data consistently improves macro-F1 by 4.2–8.7 percentage points. This work establishes a new task formulation, provides a foundational benchmark resource, and demonstrates an empirically validated pathway toward automated, polarity-aware analysis of sustainability-related text.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Humanities & Computational Social Science

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The United Nations' Sustainable Development Goals (SDGs) provide a globally recognised framework for addressing critical societal, environmental, and economic challenges. Recent developments in natural language processing (NLP) and large language models (LLMs) have facilitated the automatic classification of textual data according to their relevance to specific SDGs. Nevertheless, in many applications, it is equally important to determine the directionality of this relevance; that is, to assess whether the described impact is positive, neutral, or negative. To tackle this challenge, we propose the novel task of SDG polarity detection, which assesses whether a text segment indicates progress toward a specific SDG or conveys an intention to achieve such progress. To support research in this area, we introduce SDG-POD, a benchmark dataset designed specifically for this task, combining original and synthetically generated data. We perform a comprehensive evaluation using six state-of-the-art large LLMs, considering both zero-shot and fine-tuned configurations. Our results suggest that the task remains challenging for the current generation of LLMs. Nevertheless, some fine-tuned models, particularly QWQ-32B, achieve good performance, especially on specific Sustainable Development Goals such as SDG-9 (Industry, Innovation and Infrastructure), SDG-12 (Responsible Consumption and Production), and SDG-15 (Life on Land). Furthermore, we demonstrate that augmenting the fine-tuning dataset with synthetically generated examples yields improved model performance on this task. This result highlights the effectiveness of data enrichment techniques in addressing the challenges of this resource-constrained domain. This work advances the methodological toolkit for sustainability monitoring and provides actionable insights into the development of efficient, high-performing polarity detection systems.

Problem

• Detecting polarity of text relevance to Sustainable Development Goals
• Assessing whether text indicates positive or negative SDG impact
• Evaluating LLM performance on SDG polarity classification task

Innovation

• Proposes SDG polarity detection task for text analysis
• Introduces SDG-POD benchmark dataset with synthetic data
• Evaluates LLMs with fine-tuning and data augmentation techniques