PL-Guard: Probabilistic Logic Reasoning for LLM Guardrails

2026-08-16

AI Summary

This study addresses the safety compliance challenges in large language model guardrails arising from the coupling of semantic grounding and policy reasoning. We propose PL-Guard, a neuro-symbolic architecture that maps inputs to predicate probabilities via a local LLM and integrates ProbLog for explicit probabilistic rule inference. This approach effectively decouples the safety-utility trade-off while ensuring auditability of intermediate reasoning steps. Experiments on XSTest demonstrate that PL-Guard reduces the unsafe compliance rate to 0.5%, significantly outperforming LLM-as-a-judge baselines. However, the over-refusal rate increases to 14.4%, revealing a novel trade-off between precision and usability inherent in auditable safety guardrails. These findings highlight the potential and limitations of neuro-symbolic methods for achieving transparent, policy-compliant LLM moderation without sacrificing interpretability.

Abstract

Large language model guardrails can be viewed as policy-consistency problems: a system must determine which policy-relevant facts hold in a prompt-response pair and what those facts imply under a given policy. Common approaches, including policy prompting and LLM-as-a-judge pipelines, often overlap the tasks of semantic grounding and policy reasoning: the model both interprets the prompt-response pair and reasons about whether a policy has been violated. This can lead to unsafe compliance with harmful prompts, or refusals to assist benign ones. To separate grounding and reasoning roles, we propose PL-Guard, a neurosymbolic guardrail architecture. Using a symbolic policy interface consisting of predicates and ProbLog rules, a local LLM grounds prompt-response pairs into predicate probabilities using renormalized True/False token scores, while ProbLog performs explicit probabilistic rule inference over the symbolic policy. On the XSTest benchmark, an offline Qwen-based evaluator finds that PL-Guard with a hand-curated policy reduces unsafe compliance from 22.0% for the base model to 0.5%, and below the 6.0% rate of an LLM-as-a-judge baseline. This comes at the cost of higher over-refusal than the LLM-as-a-judge baseline, 14.4% versus 5.2%. These results suggest that separating neural grounding from probabilistic symbolic reasoning can expose the safety-helpfulness tradeoff while making the guardrail's intermediate reasoning steps explicit and auditable.

Problem

• LLM Guardrails
• Policy Consistency
• Semantic Grounding
• Safety-Helpfulness Tradeoff

Innovation

• Neurosymbolic Guardrails
• Probabilistic Logic Reasoning
• ProbLog
• Task Decoupling
• Semantic Grounding