MAETrack: Unleashing the Potential of Pretrained Geometric Priors for 3D Single Object Tracking

2026-09-15

AI Summary

本文针对3D单目标跟踪中预训练几何先验的迁移问题，提出MAETrack框架，通过分层选择性初始化和几何残差门控方法有效改善了从3D重建到3D跟踪的任务适应。

Abstract

Large-scale pre-training has transformed representation learning in 2D vision, yet its transferability to 3D single object tracking (SOT) remains insufficiently understood. Directly fine-tuning self-supervised 3D encoders, such as masked autoencoders (MAE), often leads to sub-optimal adaptation because the reconstruction objective is not fully aligned with the spatial-temporal matching requirements of tracking. In this paper, we observe that this difficulty can be interpreted as a layer-wise transfer mismatch: shallow layers tend to preserve transferable geometric cues, while deeper layers become increasingly specialized to the reconstruction pretext task and are less suitable for downstream tracking. Based on this observation, we propose MAETrack, a lightweight adaptation framework for transferring pre-training MAE representations to 3D SOT. MAETrack includes Layer-Selective Initialization (LSI), which initializes only the shallow stages of the tracking backbone from pre-trained weights while re-initializing deeper stages, and Geometric Residual Gating (GRG), which reinforces structurally salient regions in the search BEV features before template-search fusion through residual spatial modulation. Extensive experiments on standard 3D SOT benchmarks show that MAETrack consistently improves upon vanilla fine-tuning baselines with limited computational overhead. More broadly, our results suggest that effective transfer from 3D reconstruction pre-training to 3D tracking is not merely a matter of partial fine-tuning, but depends on a tracking-oriented transfer principle that preserves shallow geometry while adapting deeper representations to the downstream objective.

Problem

• 3D Single Object Tracking
• Pretrained Models
• Masked Autoencoders
• Layer-wise Transfer Mismatch
• Representation Learning

Innovation

• Layer-Selective Initialization
• Geometric Residual Gating
• 3D Single Object Tracking
• Pretrained Geometric Priors
• Transfer Learning