EdgeDAM: Real-time Object Tracking for Mobile Devices

2026-03-05

AI Summary

This work addresses the challenges of achieving high-accuracy, real-time single-object tracking on edge devices under adverse conditions such as occlusion, rapid motion, and distracting objects, which often lead to tracking drift or excessive computational cost. To this end, we propose a lightweight detection-guided tracking framework that innovatively integrates an interference-aware memory mechanism into bounding box tracking. The framework features a dual-buffer memory architecture—comprising Recent-Aware and Distractor-Resolving Memory—along with a confidence-driven strategy for switching between detection and re-identification, and a Held-Box freezing technique to stabilize bounding box estimation. Evaluated on five benchmarks including DiDi, our method achieves 88.2% accuracy on the DiDi dataset and runs at 25 FPS in real time on an iPhone 15, demonstrating a strong balance between robustness and efficiency.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Learning on the Edge & Model Compression
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Security and Privacy: Tracking, profiling, and countermeasures against them
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Single-object tracking (SOT) on edge devices is a critical computer vision task, requiring accurate and continuous target localization across video frames under occlusion, distractor interference, and fast motion. However, recent state-of-the-art distractor-aware memory mechanisms are largely built on segmentation-based trackers and rely on mask prediction and attention-driven memory updates, which introduce substantial computational overhead and limit real-time deployment on resource-constrained hardware; meanwhile, lightweight trackers sustain high throughput but are prone to drift when visually similar distractors appear. To address these challenges, we propose EdgeDAM, a lightweight detection-guided tracking framework that reformulates distractor-aware memory for bounding-box tracking under strict edge constraints. EdgeDAM introduces two key strategies: (1) Dual-Buffer Distractor-Aware Memory (DAM), which integrates a Recent-Aware Memory to preserve temporally consistent target hypotheses and a Distractor-Resolving Memory to explicitly store hard negative candidates and penalize their re-selection during recovery; and (2) Confidence-Driven Switching with Held-Box Stabilization, where tracker reliability and temporal consistency criteria adaptively activate detection and memory-guided re-identification during occlusion, while a held-box mechanism temporarily freezes and expands the estimate to suppress distractor contamination. Extensive experiments on five benchmarks, including the distractor-focused DiDi dataset, demonstrate improved robustness under occlusion and fast motion while maintaining real-time performance on mobile devices, achieving 88.2% accuracy on DiDi and 25 FPS on an iPhone 15. Code will be released.

Problem

• single-object tracking
• edge devices
• distractor interference
• real-time performance
• occlusion

Innovation

• EdgeDAM
• distractor-aware memory
• lightweight tracking
• real-time object tracking
• bounding-box tracking