AIMSCheck: Leveraging LLMs for AI-Assisted Review of Modern Slavery Statements Across Jurisdictions

2025-06-02

AI Summary

Manual compliance review of modern slavery statements across jurisdictions (e.g., UK, Canada, Australia) suffers from low efficiency, scarce labeled data, and poor generalization over legal texts. Method: We propose the first three-level decoupled automated assessment framework, integrating large language models, task decomposition, zero-shot transfer learning, and cross-domain adaptation to build an end-to-end NLP system. Contributions/Results: (1) We release AIMS.uk/AIMS.ca—the first bilingual (English–French) annotated benchmark dataset for modern slavery statement evaluation; (2) We empirically demonstrate strong cross-jurisdictional generalization of an Australian-trained model across UK and Canadian regulatory requirements; (3) Our system achieves high-accuracy compliance verification for UK and Canadian statements, with all data and code publicly released. This work significantly advances the practical deployment of RegTech in combating forced labor.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: AI for Accessibility

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Modern Slavery Acts mandate that corporations disclose their efforts to combat modern slavery, aiming to enhance transparency and strengthen practices for its eradication. However, verifying these statements remains challenging due to their complex, diversified language and the sheer number of statements that must be reviewed. The development of NLP tools to assist in this task is also difficult due to a scarcity of annotated data. Furthermore, as modern slavery transparency legislation has been introduced in several countries, the generalizability of such tools across legal jurisdictions must be studied. To address these challenges, we work with domain experts to make two key contributions. First, we present AIMS.uk and AIMS.ca, newly annotated datasets from the UK and Canada to enable cross-jurisdictional evaluation. Second, we introduce AIMSCheck, an end-to-end framework for compliance validation. AIMSCheck decomposes the compliance assessment task into three levels, enhancing interpretability and practical applicability. Our experiments show that models trained on an Australian dataset generalize well across UK and Canadian jurisdictions, demonstrating the potential for broader application in compliance monitoring. We release the benchmark datasets and AIMSCheck to the public to advance AI-adoption in compliance assessment and drive further research in this field.

Problem

• Verifying modern slavery statements is challenging due to complex language and volume.
• Developing NLP tools is difficult because of scarce annotated data.
• Generalizability of compliance tools across legal jurisdictions needs investigation.

Innovation

• Leveraging LLMs for cross-jurisdictional compliance review
• Introducing annotated datasets from UK and Canada
• End-to-end framework with three-level compliance assessment