Multi-modal Multi-platform Person Re-Identification: Benchmark and Method

2025-03-21

AI Summary

Pedestrian re-identification (ReID) under heterogeneous multi-modal (RGB/infrared/thermal) and multi-platform (UAV/ground camera) settings remains largely unexplored, posing challenges in handling sensor heterogeneity, viewpoint variation, and illumination changes. Method: We introduce MP-ReID, the first benchmark dataset for multi-platform ReID, and propose Uni-Prompt ReID—a unified framework leveraging platform-adaptive prompt learning, cross-modal feature alignment, and domain-aware contrastive training to jointly model dynamic viewpoints and heterogeneous modalities. Contribution/Results: Uni-Prompt ReID achieves significant improvements over state-of-the-art methods on MP-ReID, demonstrating robustness to illumination shifts, large viewpoint discrepancies, and sensor diversity. By enabling joint RGB-infrared-thermal representation learning, our work transcends conventional single-modal, static ReID paradigms and establishes a reproducible benchmark and novel methodology for cross-modal, cross-platform ReID in open, dynamic environments.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Conventional person re-identification (ReID) research is often limited to single-modality sensor data from static cameras, which fails to address the complexities of real-world scenarios where multi-modal signals are increasingly prevalent. For instance, consider an urban ReID system integrating stationary RGB cameras, nighttime infrared sensors, and UAVs equipped with dynamic tracking capabilities. Such systems face significant challenges due to variations in camera perspectives, lighting conditions, and sensor modalities, hindering effective person ReID. To address these challenges, we introduce the MP-ReID benchmark, a novel dataset designed specifically for multi-modality and multi-platform ReID. This benchmark uniquely compiles data from 1,930 identities across diverse modalities, including RGB, infrared, and thermal imaging, captured by both UAVs and ground-based cameras in indoor and outdoor environments. Building on this benchmark, we introduce Uni-Prompt ReID, a framework with specific-designed prompts, tailored for cross-modality and cross-platform scenarios. Our method consistently outperforms state-of-the-art approaches, establishing a robust foundation for future research in complex and dynamic ReID environments. Our dataset are available at:https://mp-reid.github.io/.

Problem

• Addressing multi-modal multi-platform person re-identification challenges
• Overcoming variations in camera perspectives and lighting conditions
• Enhancing cross-modality and cross-platform ReID performance

Innovation

• Introduces MP-ReID benchmark dataset
• Proposes Uni-Prompt ReID framework
• Supports multi-modality multi-platform scenarios