SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing

2026-06-29

AI Summary

Existing guardrail systems struggle to accurately detect policy violations according to application-specific safety policies grounded in contextual nuance. This work proposes a novel context-aware policy-based evaluation paradigm and introduces SafePyramid, a hierarchical benchmark encompassing over 60,000 natural language rules across multiple domains. SafePyramid evaluates three core capabilities: single-rule comprehension, rule-dependency reasoning, and adaptation to novel policies. Through a multi-stage human-in-the-loop construction process—integrating multi-turn dialogues, fine-grained annotations, and tiered task design—the study systematically assesses ten leading large language models and five policy guardrails. Results reveal significant limitations in contextual policy execution: even the best-performing model, GPT-5.5, achieves full accuracy rates of only 54.0%, 35.3%, and 12.9% on the three respective tiers, underscoring a critical gap in current systems’ ability to reason with and apply complex, context-dependent safety policies.

Abstract

In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on predefined risk taxonomies. In this work, we study this setting under the paradigm of in-context policy guardrailing, where guardrails predict safety violations based on policy specifications provided in context. To systematically evaluate this capability, we introduce SafePyramid, a safety benchmark comprising 1,000 multi-turn conversations across 10 domains and 3,000 corresponding application-specific policies, which together contain 61,699 distinct natural-language rules. SafePyramid organizes the evaluation into three difficulty levels: L0 evaluates individual-rule understanding, L1 evaluates reasoning over rule dependencies, and L2 evaluates adaptation of full novel policy frameworks defined in context. To ensure benchmark quality, we employ a rigorous multi-stage pipeline to construct and validate the benchmark. Using SafePyramid, we evaluate 10 frontier LLMs and 5 policy-configurable guardrails and find that in-context policy guardrailing remains highly challenging: even the best-performing model, GPT-5.5, exactly identifies the full set of violated rules in only 54.0%, 35.3%, and 12.9% cases on L0, L1, and L2, respectively. These results highlight the limitations of current guardrails and call for stronger in-context policy guardrails that can reliably execute policies, resolve rule dependencies, and adapt to novel policy frameworks.

Problem

• in-context policy guardrailing
• safety policies
• rule dependencies
• policy adaptation
• LLM safety

Innovation

• in-context policy guardrailing
• hierarchical benchmark
• rule dependency reasoning
• application-specific safety policies
• multi-turn dialogue safety