Paths: Prompt-aware Spatio-temporal Transformer with Hierarchical Multi-modal Fusion for RGB-Event Video Person Re-Identification

2026-08-13

AI Summary

This work addresses the limitations of existing RGB-event video person re-identification methods, which decouple spatial and temporal modeling and restrict multimodal fusion to a global level, thereby failing to capture fine-grained discriminative features. To overcome these challenges, the authors propose Paths, a unified framework that innovatively integrates spatial and temporal modeling within a Transformer architecture. The framework introduces a memory-augmented backbone, a prompt-aware spatiotemporal Transformer (PST), and a hierarchical multimodal fusion (HMF) mechanism to enable joint spatiotemporal modeling and multi-level fusion of RGB and event-stream data. Extensive experiments on three benchmarks—EvReID, MARS, and iLIDS-VID—demonstrate that Paths significantly outperforms current state-of-the-art methods, confirming its effectiveness and robustness.

Abstract

RGB-Event Video Person Re-Identification (RE-VReID) aims to retrieve specific person across non-overlapping cameras with complementary RGB videos and event streams. However, existing methods often decouple spatial and temporal modeling, which limits their interaction. In addition, global-level RGB-Event fusion fails to fully exploit fine-grained discriminative cues. To address these issues, we propose Paths, a unified framework with spatio-temporal modeling and hierarchical multi-modal fusion for RE-VReID. Specifically, we first design a Memory-Augmented Backbone (MAB) to maintain modality-specific identity prototypes for stable intra-modal representation learning. Then, we propose a Prompt-aware Spatio-temporal Transformer (PST) to jointly model spatial and temporal cues within a unified Transformer. Finally, we introduce a Hierarchical Multi-modal Fusion (HMF) to integrate RGB and event features at global and local levels. With these modules, our framework can learn robust and discriminative representations for RE-VReID. Extensive experiments on three public RE-VReID benchmarks including EvReID, MARS and iLIDS-VID, demonstrate the effectiveness of our proposed method. The code is available at https://github.com/Reflection0427/Paths.

Problem

• RGB-Event Video Person Re-Identification
• spatio-temporal modeling
• multi-modal fusion
• fine-grained discriminative cues

Innovation

• Prompt-aware Spatio-temporal Transformer
• Hierarchical Multi-modal Fusion
• Memory-Augmented Backbone
• RGB-Event Video Person Re-Identification
• Unified Transformer Modeling