SAPS: Shared Autonomy for Policy Steering by Blending Teleoperation with a Pretrained VLA

2026-06-13

AI Summary

This work addresses the limited robustness of pretrained vision-language-action (VLA) policies in out-of-distribution scenarios and the inefficiency and high cognitive load of pure teleoperation. To bridge this gap, the authors propose the SAPS framework, which enables shared control by dynamically fusing human teleoperation commands with pretrained VLA outputs at the action level—without requiring model retraining or architectural modifications. The core innovation lies in a cosine-similarity-based dynamic arbitration mechanism that adaptively balances control authority between human and agent. Experimental results demonstrate that SAPS improves task success rates by up to 82% in both simulation and real-robot settings, significantly reducing human intervention while accelerating task completion.

Abstract

Recent advancements in Vision-Language-Action (VLA) models have demonstrated impressive generalist capabilities in robot manipulation, yet these policies can be brittle under out-of-distribution spatial and semantic perturbations. While human teleoperation offers reliable recovery, it can demand high cognitive load and precise manual control, and existing policy steering methods often require auxiliary models or sampler modifications. In this work, we introduce Shared Autonomy for Policy Steering (SAPS), a framework that blends real-time human teleoperation commands with pretrained policy actions at the action level. SAPS requires no policy retraining, auxiliary dynamics models, or architectural modifications. We propose and evaluate three arbitration strategies to balance human and VLA policy control, including a dynamic Cosine-similarity arbitration strategy that computes the geometric agreement between human and policy actions. Across evaluations in simulation (LIBERO, LIBERO-PRO, CALVIN) and on real-world robot hardware, SAPS improves task success rates over autonomous execution by up to 82% in both simulation and the real world. Furthermore, our approach drastically reduces human intervention compared to pure teleoperation, while simultaneously achieving faster task completion times than both autonomous execution and pure teleoperation. These results demonstrate that action-level shared autonomy is a practical, model-agnostic approach for reliably deploying generalist robot policies in real-world contexts involving a human operator,with promising applications in assistive teleoperation and scalable data collection.

Problem

• Vision-Language-Action
• shared autonomy
• teleoperation
• policy steering
• out-of-distribution robustness

Innovation

• Shared Autonomy
• Policy Steering
• Vision-Language-Action Models
• Teleoperation
• Action-level Blending