Automated Evidence Extraction and Scoring for Corporate Climate Policy Engagement: A Multilingual RAG Approach

2025-09-10

AI Summary

This study addresses the challenges of time-consuming, error-prone, and linguistically limited manual assessment of corporate climate policy engagement evidence. To this end, we propose a multilingual Retrieval-Augmented Generation (RAG) framework. Methodologically, it integrates layout-aware document parsing, the Nomic multilingual embedding model, and few-shot prompting to enable precise retrieval, evidence extraction, and classification scoring of climate-related textual statements across multinational enterprises. Our key contribution is the first incorporation of document structural information alongside multilingual semantic embeddings into the RAG pipeline, significantly enhancing cross-lingual understanding robustness. Experiments demonstrate that the system achieves a 3.2× improvement in evidence extraction efficiency over baselines on multilingual corporate documents and attains an F1-score of 0.89—the current state-of-the-art. However, expert validation remains necessary for nuanced policy stance judgments in complex contextual scenarios.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

InfluenceMap's LobbyMap Platform monitors the climate policy engagement of over 500 companies and 250 industry associations, assessing each entity's support or opposition to science-based policy pathways for achieving the Paris Agreement's goal of limiting global warming to 1.5°C. Although InfluenceMap has made progress with automating key elements of the analytical workflow, a significant portion of the assessment remains manual, making it time- and labor-intensive and susceptible to human error. We propose an AI-assisted framework to accelerate the monitoring of corporate climate policy engagement by leveraging Retrieval-Augmented Generation to automate the most time-intensive extraction of relevant evidence from large-scale textual data. Our evaluation shows that a combination of layout-aware parsing, the Nomic embedding model, and few-shot prompting strategies yields the best performance in extracting and classifying evidence from multilingual corporate documents. We conclude that while the automated RAG system effectively accelerates evidence extraction, the nuanced nature of the analysis necessitates a human-in-the-loop approach where the technology augments, rather than replaces, expert judgment to ensure accuracy.

Problem

• Automating evidence extraction from corporate climate policy documents
• Reducing manual labor in assessing Paris Agreement policy alignment
• Handling multilingual data for climate engagement analysis accuracy

Innovation

• Multilingual RAG approach for evidence extraction
• Layout-aware parsing with Nomic embedding model
• Few-shot prompting for document classification