Selective Safety Steering via Value-Filtered Decoding

2026-05-14

AI Summary

This work addresses the challenge that large language models may violate safety constraints during generation, while existing decoding intervention methods often unnecessarily alter safe outputs, thereby compromising model fidelity. The authors propose a value-filtering decoding approach that intervenes only when potentially unsafe generations are detected at inference time. By incorporating a safety-value criterion, an adjustable threshold, and a dynamic token filtering mechanism, the method explicitly controls the probability of erroneous intervention. Evaluated across multiple benchmarks, this approach outperforms current baselines, achieving significantly enhanced safety while better preserving the model’s utility, fluency, and consistency with original outputs.

Abstract

While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints. A growing line of work addresses this problem by modifying the model's sampling policy at decoding time using a safety reward. However, existing decoding-time steering methods often intervene unnecessarily, modifying generations that would have been safe under the base model. Such unnecessary interventions are undesirable, as they can distort key properties of the base model such as helpfulness, fluency, style, and coherence. We propose a new test-time steering method designed to reduce such unnecessary interventions while improving the safety of unsafe responses. Our approach filters tokens using a value-based safety criterion and provides an explicit bound on the probability of false interventions. A single threshold hyperparameter controls this bound, allowing practitioners to trade off higher rates of unnecessary intervention for better output safety. Across multiple datasets and experiments, we show that our value-filtered decoding method outperforms existing baselines, achieving better trade-offs between safety, helpfulness, and similarity to the base model.

Problem

• safety steering
• large language models
• decoding-time intervention
• unnecessary modification
• safety constraints

Innovation

• value-filtered decoding
• safety steering
• decoding-time intervention
• false intervention bound
• LLM alignment