Synthetic-To-Real Video Person Re-ID

2024-02-03

AI Summary

This work addresses the reliance on costly manually annotated real-world data in cross-domain video person re-identification (re-ID). We propose an unsupervised domain adaptation framework where synthetic videos serve as the source domain and real-world videos as the target domain. Our key contributions are threefold: (1) We empirically demonstrate, for the first time, that high-fidelity synthetic videos can outperform real source domains in cross-domain transfer; (2) We design a self-supervised domain-invariant feature learning mechanism, integrating an ID-consistency loss within a Mean Teacher framework to jointly optimize static appearance and dynamic temporal features; (3) By unifying self-supervised learning, domain adaptation, and temporal modeling, our method achieves significant performance gains across five mainstream real-world video re-ID benchmarks, validating the efficacy of synthetic-to-real transfer. The code and datasets are publicly released.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Computational Photography, Image & Video Synthesis
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Person re-identification (Re-ID) is an important task and has significant applications for public security and information forensics, which has progressed rapidly with the development of deep learning. In this work, we investigate a novel and challenging setting of Re-ID, i.e., cross-domain video-based person Re-ID. Specifically, we utilize synthetic video datasets as the source domain for training and real-world videos for testing, notably reducing the reliance on expensive real data acquisition and annotation. To harness the potential of synthetic data, we first propose a self-supervised domain-invariant feature learning strategy for both static and dynamic (temporal) features. Additionally, to enhance person identification accuracy in the target domain, we propose a mean-teacher scheme incorporating a self-supervised ID consistency loss. Experimental results across five real datasets validate the rationale behind cross-synthetic-real domain adaptation and demonstrate the efficacy of our method. Notably, the discovery that synthetic data outperforms real data in the cross-domain scenario is a surprising outcome. The code and data are publicly available at https://github.com/XiangqunZhang/UDA_Video_ReID.

Problem

• Cross-domain video-based person Re-ID
• Synthetic-to-real data adaptation
• Self-supervised domain-invariant feature learning

Innovation

• Self-supervised domain-invariant feature learning
• Mean-teacher scheme with ID consistency loss
• Cross-synthetic-real domain adaptation for Re-ID