AdaTracker: Learning Adaptive In-Context Policy for Cross-Embodiment Active Visual Tracking

2026-04-22

AI Summary

Existing active visual tracking methods struggle to generalize across robot morphologies due to their reliance on platform-specific physical and dynamic characteristics, often requiring separate model training for each new embodiment. This work proposes AdaTracker, a novel framework that explicitly models “embodiment context” for the first time. AdaTracker employs an Embodiment Context Encoder to infer morphological constraints from historical interactions and dynamically modulates a context-aware tracking policy, enabling zero-shot adaptation to unseen robot embodiments. An auxiliary learning objective is introduced to enhance the accuracy and temporal consistency of context inference. Experimental results demonstrate that AdaTracker significantly outperforms current approaches in both simulation and real-world environments, achieving notable advances in cross-embodiment generalization, sample efficiency, and zero-shot adaptability.

Technology Category

• Intelligent Robots: Embodied AI
• Computer Vision: Motion & Tracking
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Realizing active visual tracking with a single unified model across diverse robots is challenging, as the physical constraints and motion dynamics vary drastically from one platform to another. Existing approaches typically train separate models for each embodiment, leading to poor scalability and limited generalization. To address this, we propose AdaTracker, an adaptive in-context policy learning framework that robustly tracks targets on diverse robot morphologies. Our key insight is to explicitly model embodiment-specific constraints through an Embodiment Context Encoder, which infers embodiment-specific constraints from history. This contextual representation dynamically modulates a Context-Aware Policy, enabling it to infer optimal control actions for unseen embodiments in a zero-shot manner. To enhance robustness, we introduce two auxiliary objectives to ensure accurate context identification and temporal consistency. Experiments in both simulation and the real world demonstrate that AdaTracker significantly outperforms state-of-the-art methods in cross-embodiment generalization, sample efficiency, and zero-shot adaptation.

Problem

• active visual tracking
• cross-embodiment
• robot generalization
• unified model
• zero-shot adaptation

Innovation

• adaptive in-context policy
• cross-embodiment generalization
• embodiment context encoder
• zero-shot adaptation
• active visual tracking