Linguistic Relative Policy Optimization for Video Anomaly Reasoning

2026-07-01

AI Summary

This work addresses the challenge of video anomaly detection under minimal human supervision, where existing methods often rely heavily on extensive annotations or expert priors. The authors propose Language-relative Policy Optimization (LRPO), a novel framework that leverages multiple reasoning trajectories to distill collective relative semantic advantages, thereby constructing both general and scene-specific linguistic priors. Guided by an anomaly-aligned reward mechanism, LRPO steers contextual reasoning to align with human risk preferences without requiring model fine-tuning. Built upon multimodal large language models, the approach integrates policy optimization, semantic distillation, and reward-guided trajectory refinement. Experiments demonstrate that LRPO significantly outperforms state-of-the-art methods on XD-Violence, UCF-Crime, and UBnormal benchmarks, achieving leading performance in a zero-fine-tuning setting.

Abstract

Video anomaly detection (VAD) with multimodal large language models has shown strong potential, yet most existing methods still depend on large-scale annotations or expert-designed priors, limiting their ability to acquire anomaly knowledge with as little human intervention as possible. To address this, we propose Linguistic Relative Policy Optimization (LRPO), which distills group-relative semantic advantages from multiple reasoning trajectories into a linguistically expressed anomaly experience prior, and adapts the model by injecting this prior into the context to steer its output distribution without any parameter updates. LRPO builds two complementary experience representations: general experience captures transferable anomaly preferences across scenarios, while scenario experience models context-dependent anomaly rules for targeted refinement. To further improve the learned experience, we introduce an anomaly alignment reward that guides trajectory optimization to match human risk preferences and reinforce temporally grounded reasoning. Extensive experiments on XD-Violence, UCF-Crime, and UBnormal demonstrate that LRPO significantly outperforms existing state-of-the-art methods under tuning-free settings.

Problem

• Video Anomaly Detection
• Multimodal Large Language Models
• Annotation Efficiency
• Anomaly Knowledge Acquisition
• Human Intervention

Innovation

• Linguistic Relative Policy Optimization
• video anomaly detection
• multimodal large language models
• experience prior
• parameter-free adaptation