SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Models

2026-09-03

AI Summary

提出了一种基于流识别和门控LoRA框架的方法，用于视觉语言模型中的令牌级安全干预，以减少对模型原始推理路径的不必要干扰。

Abstract

Existing safety alignment methods for vision-language models usually modify the model behavior globally: once the safety parameters are trained or loaded, they participate in both unsafe and already-safe generations. This always-on intervention can unnecessarily perturb the model's original reasoning path and degrade general multimodal capabilities. We argue that safety alignment should be an on-demand intervention rather than a permanent modification to every decoding trajectory. To this end, we propose a streaming recognition and gated LoRA framework for intrinsic VLM safety. During autoregressive generation, a lightweight recognizer estimates whether the current pre-token generation state is safe or unsafe. Its output updates the LoRA gate for the following decoding step; otherwise, generation follows the frozen-backbone policy. The LoRA module is trained from unsafe prefixes, transition statements, and safe continuations, so that it learns to redirect unsafe generations back to safe responses after activation. Experiments across multiple safety and general-purpose benchmarks demonstrate the effectiveness of our method in post-alignment settings.

Problem

• safety alignment
• vision-language models
• on-demand intervention
• global modification
• multimodal capabilities

Innovation

• on-demand intervention
• streaming recognition
• gated LoRA
• token-level safety
• autoregressive generation