Robot Trajectron V3: A Probabilistic Shared Control Framework for SE(3) Manipulation

2026-07-10

AI Summary

This work addresses the high cognitive load and frequent operational errors in teleoperating high-degree-of-freedom robotic arms through low-bandwidth interfaces by proposing a Bayesian inference–based shared control framework. The approach models user intent as a conditional trajectory distribution and integrates a Transformer-driven generative model, point cloud–guided candidate grasp inference, and a translation–rotation decoupled, factorized SE(3) representation to efficiently learn an intent prior over high-dimensional action spaces. Real-time Bayesian inference is then employed to generate a posterior intent estimate that actively assists control. User studies demonstrate that, compared to baseline methods, the proposed framework significantly improves task success rates and execution efficiency while substantially reducing users’ physical and mental workload.

Abstract

We aim to address the challenge of teleoperating robotic arms for high-degree-of-freedom (high-DoF) manipulation tasks, which is cognitively demanding and error-prone, particularly when relying on low-bandwidth interfaces. We propose Robot Trajectron V3 (RT-V3), a probabilistic shared control framework designed for $SE(3)$ grasping tasks. RT-V3 formulates shared control as Bayesian inference by learning a prior over user intent and combining it with real-time user commands to estimate the posterior intent distribution. The prior models user intent as a distribution over future trajectories conditioned on past robot dynamics and visual scene context. The intent prior is parameterized by a transformer-based conditional generative model that reasons over point clouds and candidate grasp poses, together with a factorized translation-rotation representation that improves learning efficiency in high-dimensional action spaces. During execution, RT-V3 continuously estimates the posterior distribution over future trajectories by combining the learned intent prior with a user-command likelihood derived from the observed control input, enabling continuous intent refinement and shared assistance. Comprehensive experiments demonstrate that RT-V3 achieves high accuracy in trajectory prediction and competitive performance in reactive planning. Furthermore, real-world user studies indicate that RT-V3 significantly outperforms baseline methods in terms of success rate and efficiency, while substantially reducing the user's physical and mental workload.

Problem

• teleoperation
• high-DoF manipulation
• shared control
• user intent
• SE(3) grasping

Innovation

• probabilistic shared control
• Bayesian inference
• transformer-based generative model
• SE(3) manipulation
• intent prediction