PapMOT: Exploring Adversarial Patch Attack Against Multiple Object Tracking

2025-04-12

European Conference on Computer Vision

AI Summary

Multi-object tracking (MOT) systems exhibit insufficient robustness against adversarial attacks, and existing digital attacks fail to generalize to physical-world deployments. Method: This paper proposes the first printable, physically deployable adversarial patch attack for MOT. It jointly optimizes detection and cross-frame identity association to generate patches that are both digitally effective and physically realizable. The method introduces a novel patch design that induces spurious detections and a temporal consistency disruption strategy to enhance perturbation transferability. Additionally, it defines new evaluation metrics tailored to MOT robustness. Contribution/Results: Extensive experiments on multiple mainstream MOT benchmarks and real-world scenarios—using printed patches captured under realistic imaging conditions—demonstrate that the attack significantly degrades IDF1 and MOTA while increasing ID switches by over 300% on average, thereby exposing critical physical-layer security vulnerabilities in MOT systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Tracking multiple objects in a continuous video stream is crucial for many computer vision tasks. It involves detecting and associating objects with their respective identities across successive frames. Despite significant progress made in multiple object tracking (MOT), recent studies have revealed the vulnerability of existing MOT methods to adversarial attacks. Nevertheless, all of these attacks belong to digital attacks that inject pixel-level noise into input images, and are therefore ineffective in physical scenarios. To fill this gap, we propose PapMOT, which can generate physical adversarial patches against MOT for both digital and physical scenarios. Besides attacking the detection mechanism, PapMOT also optimizes a printable patch that can be detected as new targets to mislead the identity association process. Moreover, we introduce a patch enhancement strategy to further degrade the temporal consistency of tracking results across video frames, resulting in more aggressive attacks. We further develop new evaluation metrics to assess the robustness of MOT against such attacks. Extensive evaluations on multiple datasets demonstrate that our PapMOT can successfully attack various architectures of MOT trackers in digital scenarios. We also validate the effectiveness of PapMOT for physical attacks by deploying printed adversarial patches in the real world.

Problem

• Exploring adversarial patch attacks on multiple object tracking
• Generating physical patches to mislead detection and identity association
• Evaluating MOT robustness against digital and physical adversarial attacks

Innovation

• Generates physical adversarial patches for MOT
• Optimizes printable patches to mislead tracking
• Enhances patches to degrade temporal consistency