TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesis

2025-05-30

AI Summary

Large language models (LLMs) are prone to generating harmful content or being maliciously exploited; existing safety alignment datasets suffer from narrow risk coverage—primarily emphasizing lexical diversity while neglecting critical dimensions such as malicious intent and jailbreaking strategies. Method: We propose the first red-teaming data synthesis framework spanning three orthogonal dimensions: lexical diversity, malicious intent, and jailbreaking strategies. It introduces the first systematic, quantitative evaluation of risk coverage in alignment datasets and designs a role-playing–based zero-shot automated red-teaming paradigm to generate harmful instructions paired with ethically aligned responses. Contribution/Results: Experiments on Llama 3.1-8B show that supervised fine-tuning using our synthesized data reduces average harm scores by 14.29% and attack success rates by 20%, significantly outperforming the WildBreak baseline.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Humans and AI: Human-AI Collaboration / Human-AI Teaming

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) excel in various natural language processing tasks but remain vulnerable to generating harmful content or being exploited for malicious purposes. Although safety alignment datasets have been introduced to mitigate such risks through supervised fine-tuning (SFT), these datasets often lack comprehensive risk coverage. Most existing datasets focus primarily on lexical diversity while neglecting other critical dimensions. To address this limitation, we propose a novel analysis framework to systematically measure the risk coverage of alignment datasets across three essential dimensions: Lexical Diversity, Malicious Intent, and Jailbreak Tactics. We further introduce TRIDENT, an automated pipeline that leverages persona-based, zero-shot LLM generation to produce diverse and comprehensive instructions spanning these dimensions. Each harmful instruction is paired with an ethically aligned response, resulting in two datasets: TRIDENT-Core, comprising 26,311 examples, and TRIDENT-Edge, with 18,773 examples. Fine-tuning Llama 3.1-8B on TRIDENT-Edge demonstrates substantial improvements, achieving an average 14.29% reduction in Harm Score, and a 20% decrease in Attack Success Rate compared to the best-performing baseline model fine-tuned on the WildBreak dataset.

Problem

• Enhancing LLM safety by addressing harmful content generation
• Improving risk coverage in safety alignment datasets
• Automating diverse red-teaming data synthesis across key dimensions

Innovation

• Tri-dimensional analysis for risk coverage
• Automated pipeline for diverse instruction generation
• Persona-based zero-shot LLM generation