Large language models in climate and sustainability policy: limits and opportunities

2025-02-04

AI Summary

Climate and sustainability policy texts exhibit high heterogeneity, rendering manual analysis inefficient and error-prone. Method: We propose an end-to-end NLP framework integrating static fine-tuning and prompt engineering to rigorously evaluate the real-world efficacy of both general-purpose (e.g., LLaMA, GPT) and domain-adapted large language models (LLMs) within authentic policy workflows. Contribution/Results: This work presents the first empirical identification of three critical bottlenecks—knowledge integration bias, insufficient interpretability, and a human intervention threshold—thereby establishing a human-AI co-optimization pathway. Experiments demonstrate ≥85% accuracy on multi-source policy text classification and summarization tasks. The framework delivers a reproducible methodological benchmark for AI-augmented computational social science and provides actionable, practice-oriented guidance for policy implementation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Application Domains: Humanities & Computational Social Science

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

As multiple crises threaten the sustainability of our societies and pose at risk the planetary boundaries, complex challenges require timely, updated, and usable information. Natural-language processing (NLP) tools enhance and expand data collection and processing and knowledge utilization capabilities to support the definition of an inclusive, sustainable future. In this work, we apply different NLP techniques, tools and approaches to climate and sustainability documents to derive policy-relevant and actionable measures. We focus on general and domain-specific large language models (LLMs) using a combination of static and prompt-based methods. We find that the use of LLMs is successful at processing, classifying and summarizing heterogeneous text-based data. However, we also encounter challenges related to human intervention across different workflow stages and knowledge utilization for policy processes. Our work presents a critical but empirically grounded application of LLMs to complex policy problems and suggests avenues to further expand Artificial Intelligence-powered computational social sciences.

Problem

• Assessing LLMs in climate policy
• Enhancing data for sustainability
• Addressing NLP challenges in policy processes

Innovation

• NLP techniques for policy
• LLMs for text processing
• Prompt-based methods application