Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steering

2026-06-15

AI Summary

This work addresses the vulnerability of open-weight video diffusion models to generating harmful content such as violence or misinformation. Existing safety interventions either require costly fine-tuning that compromises model generality or rely on external filters that are easily circumvented. To overcome these limitations, the authors propose a training-free, inference-stage alignment method that leverages Supervised Principal Component Analysis (Supervised PCA) to identify safety directions in the latent space. A lightweight vector intervention is then applied to intermediate Transformer hidden states, steering potentially harmful generation trajectories toward semantically consistent and safe outputs. The approach demonstrates consistent effectiveness across nine prominent video diffusion models (ranging from 1.3B to 5B parameters) and generalizes to both text-to-video and image-to-video tasks, imposing negligible computational overhead while preserving the model’s broad generative capabilities.

Abstract

Open-weight video diffusion models can generate photorealistic unsafe content, from violence to misinformation, yet existing defenses either require expensive safety fine-tuning that degrades general capability, or apply external filters that are trivially bypassed by adversarial prompts. We present REINS (REpresentation-space INference-time Safety steering), a training-free method that aligns video diffusion models at inference time by steering their internal representations toward safe generation. Our key finding is that safety-relevant structure is linearly encoded in the hidden-state activations of video diffusion transformers, and a single direction, discovered via Supervised PCA on binary safety labels, suffices to separate safe from unsafe generation trajectories. At inference, adding this direction to hidden states at an intermediate transformer layer redirects generation from harmful content to semantically related safe alternatives, with no weight updates, no concept enumeration, and negligible computational overhead. Through mechanistic analysis, we reveal that while safety information accumulates monotonically with transformer depth, steering effectiveness peaks at intermediate layers (~50% depth), exposing a fundamental tradeoff between information availability and downstream propagation capacity. We evaluate REINS across 9 video diffusion models, multiple parameter scales (1.3B-5B), and both text-to-video and image-to-video generation, to our knowledge, the broadest safety evaluation suite in the video generation literature.

Problem

• video diffusion models
• safety alignment
• unsafe content
• adversarial prompts
• training-free

Innovation

• representation steering
• training-free safety alignment
• video diffusion models
• inference-time intervention
• Supervised PCA