LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search

2026-08-10

AI Summary

Existing text-driven pedestrian anomaly search methods struggle to disentangle appearance and motion features, often suffering from visual entanglement and shortcut learning induced by hard negative mining in Euclidean space. To address this, this work proposes a lightweight motion inversion operator to extract clean motion representations and enforces strict forward disentanglement via orthogonal null-space projection. Furthermore, a Riemannian gradient correction module is introduced to constrain the backward propagation path, preventing degradation of disentanglement properties. Training stability is enhanced by integrating textual semantic priors with tangent space optimization. Notably, the method operates without external pose estimators and achieves significant performance gains over state-of-the-art approaches on both TPAS and TIPR benchmarks, demonstrating robustness and effectiveness in complex surveillance scenarios.

Abstract

Traditional Text-based Person Search (TPS) is typically limited to matching static appearance attributes, severely neglecting dynamic action information. The Text-based Person Anomaly Search (TPAS) task bridges this gap, requiring models to locate micro-level specific abnormal behaviors while matching macro-level appearance of pedestrians. However, current TPAS methods face fundamental limitations: external explicit pose estimators are fragile in unconstrained surveillance scenarios, and implicit learning encounters visual decoupling failure under pixel-level entanglement, causing dominant appearance information to easily swallow and contaminate subtle action features. Furthermore, performing contrastive optimization on hard negative samples (``same appearance, different actions'') in conventional Euclidean spaces induces severe shortcut learning. To address these, we propose the Lightweight Action Inversion and Riemannian rectification network (LightAIR). First, it introduces textual semantic priors as anchors via a lightweight action inversion operator to extract pure action features, thereby overcoming visual-inherent coupling. Subsequently, it employs orthogonal null-space projection to constrain appearance features within the orthogonal complement space of action features, guaranteeing strict forward decoupling. Finally, we designed a gradient rectification module that computes the Riemannian gradient to constrain the backpropagation trajectory, forcing the gradient flow to update strictly along the tangent space that preserves decoupling properties, thereby cutting off harmful shortcuts. Extensive experiments on the widely used TPAS and TIPR datasets demonstrate that LightAIR significantly outperforms existing state-of-the-art methods. Codes are available at https://github.com/rainy-london/LightAIR

Problem

• Text-based Person Anomaly Search
• action-appearance decoupling
• visual feature entanglement
• shortcut learning
• hard negative samples

Innovation

• action inversion
• Riemannian rectification
• feature decoupling
• text-based person anomaly search
• orthogonal null-space projection