Test-Time Detoxification without Training or Learning Anything

2026-01-14

arXiv.org

AI Summary

Large language models can generate toxic content even when given harmless inputs, posing safety risks and undermining user trust. This work proposes a test-time detoxification method that, for the first time, treats word embeddings as controllable variables. By employing zeroth-order optimization to estimate the gradient of toxicity with respect to input embeddings, the approach requires only forward passes through the model, access to input embeddings, and a toxicity scoring function—without any model training or internal access. Applicable to black-box models, the method significantly reduces output toxicity across diverse models and prompt settings while achieving the current best trade-off between toxicity suppression and text quality.

Abstract

Large language models can produce toxic or inappropriate text even for benign inputs, creating risks when deployed at scale. Detoxification is therefore important for safety and user trust, particularly when we want to reduce harmful content without sacrificing the model's generation quality. Many existing approaches rely on model retraining, gradients, or learned auxiliary components, which can be costly and may not transfer across model families or to truly black-box settings. We introduce a test-time procedure that approximates the gradient of completion toxicity with respect to the input embeddings and uses a small number of descent steps to steer generation toward less toxic continuations. This is achieved with zeroth-order optimization that requires only access to input embeddings, a toxicity scoring function, and forward evaluations of the model. Empirically, the approach delivers robust toxicity reductions across models and prompts and, in most settings, achieves the best overall toxicity-quality trade-off. More broadly, our work positions word embeddings as effective control variables and encourages wider use of black-box optimization to guide autoregressive language models toward scalable, safer text generation, without requiring any training or access to intermediate computations.

Problem

• toxicity
• large language models
• detoxification
• safety
• black-box

Innovation

• test-time detoxification
• zeroth-order optimization
• black-box language models
• embedding steering
• toxicity reduction