CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

2026-06-13

AI Summary

This work addresses the inadequacy of existing safety guardrails for large language models in Chinese contexts, where challenges arise from misalignment with regulatory policies, cultural nuances, and linguistic subtleties, as well as a lack of fine-grained risk categorization. To tackle these issues, we propose CHILLGuard—a Chinese-oriented, fine-grained content safety guardrail—featuring a comprehensive risk taxonomy spanning five major categories and thirty-one subcategories. We introduce an innovative pipeline that integrates retrieval-augmented generation, prompt rewriting, and multi-model voting calibration to construct high-quality training data. Furthermore, we develop a model-aware direct preference optimization (Model-aware DPO) approach alongside a generator-classifier co-training framework. Evaluated on our newly curated CHILLGuardTrain (405,007 samples) and CHILLGuardTest (51,745 samples) datasets, CHILLGuard achieves state-of-the-art performance, outperforming Qwen3Guard-8B-Strict by 15.92% in F1 score on Chinese safety benchmarks.

Abstract

Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns. While existing LLM safety guardrails excel in English or multilingual settings, they lack adaptation to Chinese-specific regulatory policies, cultural context and linguistic nuances, failing to support fine-grained risk classification for diverse deployment needs. In this paper, we introduce a 5-macro, 31-micro category fine-grained risk taxonomy for Chinese scenarios, and build CHILLGuard: a dedicated Chinese LLM content safety guardrail. To address the critical scarcity of high-quality annotated Chinese safety data, we propose a scalable multi-stage data construction pipeline: we expand multi-source corpus via retrieval-augmented generation, generate implicit harmful samples through prompt engineering rewriting, and refine high-quality data via multi-model voting-based label calibration. Based on this, we build CHILLGuardTrain, a large-scale training set with 405,007 samples, and CHILLGuardTest, a rigorously curated annotated test set with 51,745 samples. We then train CHILLGuard on CHILLGuardTrain under a generator-classifier collaborative framework via Model-aware Direct Preference Optimization. Extensive experiments under multiple settings demonstrate the state-of-the-art performance of CHILLGuard, e.g., a 15.92% improvement of F1 score over Qwen3Guard-8B-Strict on our benchmark. We will release our resources at https://github.com/cswbyu/CHILLGuard.

Problem

• Chinese LLM safety
• fine-grained risk classification
• content moderation
• linguistic nuances
• regulatory compliance

Innovation

• fine-grained safety taxonomy
• scalable data construction
• model-aware preference alignment
• Chinese LLM guardrail
• retrieval-augmented generation