SPIRIT: Perceptive Shared Autonomy for Robust Robotic Manipulation under Deep Learning Uncertainty
🤖 AI Summary
Deep learning–based perception in robotics often struggles to meet the demands of safety-critical applications due to limited robustness and poor interpretability. This work proposes a perception-driven shared autonomy framework that, for the first time, leverages perceptual uncertainty to dynamically modulate control authority between human and robot. Specifically, when the confidence of neural tangent kernel (NTK)–based point cloud registration is high, the system enables semi-autonomous operation to enhance performance; as uncertainty increases, it seamlessly transitions to haptic teleoperation to ensure robustness. Evaluated in a user study involving 15 participants performing aerial manipulation tasks and in industrial simulations, the approach significantly improves both task performance and system reliability, earning a finalist position in a major industrial innovation award.
Application Category
📝 Abstract
Deep learning (DL) has enabled impressive advances in robotic perception, yet its limited robustness and lack of interpretability hinder reliable deployment in safety critical applications. We propose a concept termed perceptive shared autonomy, in which uncertainty estimates from DL based perception are used to regulate the level of autonomy. Specifically, when the robot's perception is confident, semi-autonomous manipulation is enabled to improve performance; when uncertainty increases, control transitions to haptic teleoperation for maintaining robustness. In this way, high-performing but uninterpretable DL methods can be integrated safely into robotic systems. A key technical enabler is an uncertainty aware DL based point cloud registration approach based on the so called Neural Tangent Kernels (NTK). We evaluate perceptive shared autonomy on challenging aerial manipulation tasks through a user study of 15 participants and realization of mock-up industrial scenarios, demonstrating reliable robotic manipulation despite failures in DL based perception. The resulting system, named SPIRIT, improves both manipulation performance and system reliability. SPIRIT was selected as a finalist of a major industrial innovation award.
Problem
Research questions and friction points this paper is trying to address.
deep learning uncertainty
robotic manipulation
shared autonomy
safety-critical applications
perception reliability
Innovation
Methods, ideas, or system contributions that make the work stand out.
perceptive shared autonomy
uncertainty-aware deep learning
Neural Tangent Kernels
haptic teleoperation
robust robotic manipulation
🔎 Similar Papers
No similar papers found.
