Learning to Track from Privileged Target Appearances

2026-09-02

AI Summary

该研究通过引入一种名为PATT的教师-学生训练框架，利用不可用的当前及未来帧目标裁剪来改善视觉跟踪器性能，从而解决目标模板随时间变旧的问题。

Abstract

Target templates define what a visual tracker searches for, yet the templates available at inference trade off localization certainty with appearance freshness: the initial ground-truth template is exact but becomes stale, whereas recent templates better reflect the current appearance but are cropped from uncertain predictions. We quantify this bottleneck with a non-deployable oracle that supplies an exact current-frame target crop, improving AUC on LaSOT by 15.2 percentage points. This gap reveals a training-only opportunity: frame-level ground truths provide exact current- and future-frame target crops, although such crops are unavailable at deployment. We introduce Privileged Appearance Transfer for Tracking (PATT), a teacher-student training framework that transfers these privileged appearances to a deployable tracker through multi-level representation prediction. The privileged teacher observes exact target crops from past, current, and future frames, whereas the student receives only past-frame templates and learns to predict the teacher's search representations. To avoid transferring unreliable teacher signals, PATT weights this transfer by the teacher's relative localization advantage over the student and its absolute localization accuracy. After training, the teacher, latent predictor, reliability weights, and privileged crops are removed, leaving standard student-only inference. Across seven benchmarks at two model scales, PATT achieves consistent gains under both long- and short-term tracking protocols.

Problem

• visual tracker
• target template
• localization certainty
• appearance freshness
• inference

Innovation

• Privileged Appearance Transfer
• teacher-student training framework
• multi-level representation prediction
• localization accuracy