A Comparative Study of Task Adaptation Techniques of Large Language Models for Identifying Sustainable Development Goals

2025-06-18

AI Summary

Manual tracking of Sustainable Development Goal (SDG)-related texts is hindered by their large scale and semantic complexity. Method: This study systematically evaluates the adaptation efficacy of mainstream large language models (LLMs) for single-label, multi-class SDG text classification, employing prompt engineering, zero-shot and few-shot learning, and lightweight fine-tuning across both open-source and commercial models. Contribution/Results: Optimized small-scale open-source LLMs—specifically Phi-3 and Qwen2—achieve classification accuracy comparable to or exceeding that of GPT-4, with up to a 12.3% absolute improvement in SDG label accuracy. These findings challenge the prevailing “bigger is better” assumption in LLM deployment and empirically validate lightweight adaptation as a viable strategy. The approach delivers an efficient, deployable AI solution for SDG monitoring in resource-constrained settings, balancing performance, computational efficiency, and accessibility.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

In 2012, the United Nations introduced 17 Sustainable Development Goals (SDGs) aimed at creating a more sustainable and improved future by 2030. However, tracking progress toward these goals is difficult because of the extensive scale and complexity of the data involved. Text classification models have become vital tools in this area, automating the analysis of vast amounts of text from a variety of sources. Additionally, large language models (LLMs) have recently proven indispensable for many natural language processing tasks, including text classification, thanks to their ability to recognize complex linguistic patterns and semantics. This study analyzes various proprietary and open-source LLMs for a single-label, multi-class text classification task focused on the SDGs. Then, it also evaluates the effectiveness of task adaptation techniques (i.e., in-context learning approaches), namely Zero-Shot and Few-Shot Learning, as well as Fine-Tuning within this domain. The results reveal that smaller models, when optimized through prompt engineering, can perform on par with larger models like OpenAI's GPT (Generative Pre-trained Transformer).

Problem

• Identifying Sustainable Development Goals in text data
• Comparing task adaptation techniques for LLMs
• Evaluating performance of small vs large language models

Innovation

• Evaluates task adaptation techniques for LLMs
• Compares Zero-Shot and Few-Shot Learning methods
• Optimizes smaller models via prompt engineering