Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Information

2025-12-10

AI Summary

This paper introduces Hierarchical Instance Tracking (HIT), a novel task that jointly tracks object-level and part-level instances while explicitly modeling semantic hierarchical relationships—balancing privacy preservation with fine-grained visual accessibility. To formalize HIT, we provide the first rigorous task definition and present HIT-552, the first large-scale benchmark comprising 552 videos, 2,765 annotated entities, and 40 categories spanning objects and parts; it is officially adopted by VizWiz as a standard challenge. Methodologically, we extend seven state-of-the-art multi-object tracking frameworks—including Mask R-CNN and TrackFormer—with hierarchical-aware association and mask propagation mechanisms. Extensive experiments reveal substantial performance degradation of existing models on HIT-552, confirming the task’s inherent difficulty. Both the benchmark and all baseline implementations are fully open-sourced to advance research in fine-grained visual tracking.

Technology Category

• Computer Vision: Motion & Tracking
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Security and Privacy: Tracking, profiling, and countermeasures against them
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Topic discovery and tracking

Abstract

We propose a novel task, hierarchical instance tracking, which entails tracking all instances of predefined categories of objects and parts, while maintaining their hierarchical relationships. We introduce the first benchmark dataset supporting this task, consisting of 2,765 unique entities that are tracked in 552 videos and belong to 40 categories (across objects and parts). Evaluation of seven variants of four models tailored to our novel task reveals the new dataset is challenging. Our dataset is available at https://vizwiz.org/tasks-and-datasets/hierarchical-instance-tracking/

Problem

• Track hierarchical object and part instances in videos
• Introduce a benchmark dataset with 2765 entities
• Evaluate models on the challenging new tracking task

Innovation

• Hierarchical instance tracking for object and part categories
• First benchmark dataset with 2,765 entities in 552 videos
• Evaluation of seven model variants tailored to the task