Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliance

2025-09-26

AI Summary

Existing LLM safety approaches rely on empirical categorization, lacking systematicity and formal verifiability, and thus struggle to address complex, fine-grained risks. This paper introduces the first legal-compliance-driven LLM safety paradigm, anchored in the EU AI Act and GDPR, to establish an interpretable, quantifiable, and standardized safety benchmark and evaluation framework. We propose a legal-provision-guided scenario generation method and design Group-based Reward Policy Optimization (GRPO), a novel alignment algorithm, to train Qwen3-8B into a legally grounded reasoning agent capable of compliance-aware inference. Evaluated on our proprietary compliance benchmark, the aligned model achieves average accuracy improvements of 10.45% on AI Act–related tasks and 11.85% on GDPR-related tasks, significantly enhancing the legal consistency and auditability of safety-critical decisions.

Technology Category

• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The proliferation of Large Language Models (LLMs) has demonstrated remarkable capabilities, elevating the critical importance of LLM safety. However, existing safety methods rely on ad-hoc taxonomy and lack a rigorous, systematic protection, failing to ensure safety for the nuanced and complex behaviors of modern LLM systems. To address this problem, we solve LLM safety from legal compliance perspectives, named safety compliance. In this work, we posit relevant established legal frameworks as safety standards for defining and measuring safety compliance, including the EU AI Act and GDPR, which serve as core legal frameworks for AI safety and data security in Europe. To bridge the gap between LLM safety and legal compliance, we first develop a new benchmark for safety compliance by generating realistic LLM safety scenarios seeded with legal statutes. Subsequently, we align Qwen3-8B using Group Policy Optimization (GRPO) to construct a safety reasoner, Compliance Reasoner, which effectively aligns LLMs with legal standards to mitigate safety risks. Our comprehensive experiments demonstrate that the Compliance Reasoner achieves superior performance on the new benchmark, with average improvements of +10.45% for the EU AI Act and +11.85% for GDPR.

Problem

• Addressing inadequate systematic safety protection in LLMs
• Aligning LLM safety with established legal compliance frameworks
• Developing benchmark and reasoner for legal standard compliance

Innovation

• Aligning LLMs with legal compliance frameworks
• Developing safety benchmark using legal statutes
• Using Group Policy Optimization for safety reasoning