ML-Bench&Guard: Policy-Grounded Multilingual Safety Benchmark and Guardrail for Large Language Models

2026-05-01

AI Summary

Existing multilingual safety evaluation benchmarks for large language models predominantly rely on generic risk taxonomies and machine translation, which struggle to accommodate regional legal frameworks and cultural nuances. To address this limitation, this work introduces ML-Bench—the first multilingual safety benchmark grounded in jurisdiction-specific legal texts—and presents ML-Guard, a diffusion-based large language model (dLLM)–driven defense system capable of rapid safety judgments and fine-grained compliance explanations across 14 languages. By leveraging regionally regulated data generation and employing both lightweight and high-capacity conditional defense strategies, ML-Guard substantially outperforms 11 strong baselines across six established benchmarks as well as the newly constructed ML-Bench, significantly enhancing safety enforcement and compliance assessment in multilingual settings.

Abstract

As Large Language Models (LLMs) are increasingly deployed in cross-linguistic contexts, ensuring safety in diverse regulatory and cultural environments has become a critical challenge. However, existing multilingual benchmarks largely rely on general risk taxonomies and machine translation, which confines guardrail models to these predefined categories and hinders their ability to align with region-specific regulations and cultural nuances. To bridge these gaps, we introduce ML-Bench, a policy-grounded multilingual safety benchmark covering 14 languages. ML-Bench is constructed directly from regional regulations, where risk categories and fine-grained rules derived from jurisdiction-specific legal texts are directly used to guide the generation of multilingual safety data, enabling culturally and legally aligned evaluation across languages. Building on ML-Bench, we develop ML-Guard, a Diffusion Large Language Model (dLLM)-based guardrail model that supports multilingual safety judgment and policy-conditioned compliance assessment. ML-Guard has two variants, one 1.5B lightweight model for fast `safe/unsafe' checking and a more capable 7B model for customized compliance checking with detailed explanations. We conduct extensive experiments against 11 strong guardrail baselines across 6 existing multilingual safety benchmarks and our ML-Bench, and show that ML-Guard consistently outperforms prior methods. We hope that ML-Bench and ML-Guard can help advance the development of regulation-aware and culturally aligned multilingual guardrail systems.

Problem

• multilingual safety
• large language models
• regulatory alignment
• cultural nuance
• safety benchmark

Innovation

• policy-grounded benchmark
• multilingual safety
• Diffusion LLM
• regulation-aware guardrail
• culturally aligned evaluation