When in Doubt, Cascade: Towards Building Efficient and Capable Guardrails

2024-07-08

arXiv.org

AI Summary

To address harmfulness and bias in large language model (LLM) outputs, this work proposes a cascaded output guard framework grounded in the cognitive distinction between *use* (user intent) and *description* (system behavior). Methodologically, it introduces the first decoupled modeling of use and description, enabling a taxonomy-driven instruction engineering and synthetic data generation pipeline that produces over 300K high-quality contrastive samples. Leveraging contrastive learning and a lightweight cascaded detection architecture, the framework achieves efficient, reproducible guard modeling. Evaluated on multiple open-source benchmarks, it attains state-of-the-art performance while reducing inference overhead by over 40%. This work establishes a scalable, interpretable, and low-resource-dependent methodology for LLM safety alignment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) have convincing performance in a variety of downstream tasks. However, these systems are prone to generating undesirable outputs such as harmful and biased text. In order to remedy such generations, the development of guardrail (or detector) models has gained traction. Motivated by findings from developing a detector for social bias, we adopt the notion of a use-mention distinction - which we identified as the primary source of under-performance in the preliminary versions of our social bias detector. Armed with this information, we describe a fully extensible and reproducible synthetic data generation pipeline which leverages taxonomy-driven instructions to create targeted and labeled data. Using this pipeline, we generate over 300K unique contrastive samples and provide extensive experiments to systematically evaluate performance on a suite of open source datasets. We show that our method achieves competitive performance with a fraction of the cost in compute and offers insight into iteratively developing efficient and capable guardrail models. Warning: This paper contains examples of text which are toxic, biased, and potentially harmful.

Problem

• Detect harmful biased text in LLM outputs
• Improve guardrail model efficiency and capability
• Generate synthetic data for detector training

Innovation

• Use-mention distinction improves detector performance
• Taxonomy-driven synthetic data generation pipeline
• Generates 300K contrastive samples cost-effectively