Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking

2026-06-24

AI Summary

This work addresses the degraded robustness in RGB-event visual object tracking caused by modality degradation and partial target loss—such as occlusion or truncation—by proposing the APRTrack framework. The method introduces structured adversarial perturbations to simulate multimodal signal degradation and employs a hierarchical perturbation decoupling training strategy. Additionally, it incorporates a footprint-guided channel-calibrated Hopfield memory retrieval mechanism (FCHR) to enable controllable compensation of historical features. Through an integrated architecture combining adversarial perturbation branches, hierarchical routing, and multimodal fusion, APRTrack significantly enhances recovery capability for partially missing targets. Extensive experiments on FE108, COESOT, VisEvent, and FELT benchmarks demonstrate state-of-the-art performance, validating the effectiveness of the proposed approach under complex degradation scenarios.

Abstract

RGB-Event tracking improves localization robustness by fusing RGB appearance textures and dense temporal motion cues from event sensors. While this multi-modal scheme broadens tracking applicability, real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion. In harsh environments, either modality can lose reliability drastically, and targets frequently appear incomplete due to occlusion, edge truncation and foreground clutter.To tackle the above challenges, we present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation, termed APRTrack. To mimic real-world signal corruption, APRTrack constructs structured degradation via two adversarial perturbation branches at the modality and spatial levels, which separately simulate full-modal failure and localized target region absence. A hierarchical routing mechanism is designed to disentangle the training pipelines of the two perturbation types, effectively eliminating feature collapse induced by superimposed degradation constraints. Furthermore, we devise Footprint-guided Channel-calibrated Hopfield Retrieval (FCHR) for reliable historical information compensation. This module evaluates retrieval confidence based on association footprints between queries and memory banks, and calibrates the retrieval metric space prior to Hopfield matching, realizing controllable historical feature compensation bounded to target regions. Extensive experiments on FE108, COESOT, VisEvent, and FELT datasets demonstrate the effectiveness of our proposed strategies for the RGB-Event visual object tracking. The source code and pre-trained models will be released on https://github.com/Event-AHU/OpenEvTracking

Problem

• RGB-Event tracking
• structured degradation
• modal reliability
• partial target missing
• multi-modal fusion

Innovation

• adversarial perturbation
• associative memory retrieval
• RGB-event tracking
• structured degradation
• Hopfield network