Adaptor: Advancing Assistive Teleoperation with Few-Shot Learning and Cross-Operator Generalization

2026-04-10

AI Summary

This work addresses the heterogeneity in trajectory distributions caused by varying operator skill levels in shared control settings. To tackle this challenge, the authors propose the Adaptor framework, which models intention uncertainty and extracts geometry-aware keyframes to fuse contextual information from an intention expert and a pretrained vision-language model during preprocessing, thereby guiding an action expert to generate robust policies. Innovatively integrating few-shot learning with cross-operator generalization, the method enables rapid adaptation to new operators while maintaining low performance variance. The approach encompasses trajectory perturbation synthesis, keyframe extraction, intention encoding, and conditional action generation. Evaluated in both real-world and simulated environments, it significantly improves task success rates and operational efficiency, demonstrating strong generalization across diverse skill levels.

Technology Category

• Multiagent Systems: Adversarial Agents
• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Assistive teleoperation enhances efficiency via shared control, yet inter-operator variability, stemming from diverse habits and expertise, induces highly heterogeneous trajectory distributions that undermine intent recognition stability. We present Adaptor, a few-shot framework for robust cross-operator intent recognition. The Adaptor bridges the domain gap through two stages: (i) preprocessing, which models intent uncertainty by synthesizing trajectory perturbations via noise injection and performs geometry-aware keyframe extraction; and (ii) policy learning, which encodes the processed trajectories with an Intention Expert and fuses them with the pre-trained vision-language model context to condition an Action Expert for action generation. Experiments on real-world and simulated benchmarks demonstrate that Adaptor achieves state-of-the-art performance, improving success rates and efficiency over baselines. Moreover, the method exhibits low variance across operators with varying expertise, demonstrating robust cross-operator generalization.

Problem

• assistive teleoperation
• intent recognition
• inter-operator variability
• trajectory heterogeneity
• cross-operator generalization

Innovation

• few-shot learning
• cross-operator generalization
• assistive teleoperation
• intent recognition
• vision-language model