HITL-D: Human In The Loop Diffusion Assisted Shared Control

2026-05-20

AI Summary

This work addresses the complexity of human–robot interaction in multi-step, insertion-based, and fine teleoperation tasks by proposing a human-in-the-loop shared control framework that, for the first time, integrates diffusion policies into teleoperation systems. The approach combines human input with a point-cloud-based diffusion model to automatically adjust the end-effector orientation of a robotic arm, enabling high-dimensional manipulation through position-only control and thereby significantly simplifying the operator interface. Experimental results demonstrate that, compared to conventional methods, the proposed framework reduces average task completion time by 40% and subjective workload by 37%, while substantially improving perceived intuitiveness, user autonomy, and confidence in system performance.

Abstract

Autonomous manipulation systems have achieved remarkable capabilities, yet the integration of human expertise with diffusion-based policies in shared control remains relatively unexplored. In this paper, we propose Human-In-The-Loop Diffusion (HITL-D), a shared control framework that enhances user performance in multi-step, insertion, and fine manipulation tasks. HITL-D leverages a novel combination of diffusion-based policies and human control to provide autonomous end effector orientation updates conditioned on a scene point cloud and the Cartesian position of the end effector. This approach reduces the number of joystick control axes required, thereby lowering mental workload. In a multi-task user study with 12 participants, HITL-D reduced average task completion times by 40%, decreased perceived workload by 37%, and improved Likert-scale ratings for independence, intuitiveness, and confidence compared to traditional teleoperation methods. These results demonstrate that HITL-D effectively integrates human expertise with autonomous assistance, improving both objective and subjective aspects of teleoperation.

Problem

• shared control
• human-in-the-loop
• diffusion policies
• teleoperation
• autonomous manipulation

Innovation

• diffusion-based policy
• shared control
• human-in-the-loop
• teleoperation
• autonomous manipulation