Flying Together: Human-Guided Immersive Shared Control for Aerial Robot Teams in Unknown Environments

2026-05-20

AI Summary

This work addresses the challenges faced by autonomous multi-robot systems in unknown, unstructured environments, where unexpected situations arise and operator intent is difficult to interpret accurately. The authors propose a virtual reality–based shared control framework that integrates a user-guided motion primitive planner with an admittance controller, enabling immersive, real-time human guidance of drone swarms. The approach introduces a novel mechanism for generating continuous, collision-free trajectories informed by user input, supports mixed-reality operation with bidirectional VR interaction, and incorporates a waypoint guidance strategy to direct drones toward regions commonly overlooked by fully autonomous systems. Experimental results demonstrate that the framework significantly enhances obstacle avoidance performance, maintains safe inter-agent distances, and reduces operator cognitive workload, thereby validating the feasibility and advantages of human-in-the-loop multi-drone navigation.

Abstract

While autonomous multi-robots can achieve safe and coordinated navigation, they often struggle to adapt to unforeseen conditions and to capture operator-driven objectives in unstructured environments. We present a Virtual Reality (VR)-based shared control framework for teams of drones operating in constrained and unknown environments, enabling real-time, user-guided exploration. At the core of our approach is a novel, user-guided motion-primitive-based planner that computes continuous, collision-free trajectories while continuously integrating operator input. This planner is coupled with an admittance controller, allowing the operator to flexibly influence team behavior and guide drones toward regions of interest that autonomous planners may overlook. The system supports mixed-reality operations with both physical and simulated drones, and implements a bilateral VR-based interface, allowing the operator to guide the robot team via migration points while receiving immediate visual feedback of the team state. Experimental results show that shared control improves obstacle avoidance, maintains inter-agent spacing, and reduces operator effort, demonstrating the feasibility and advantages of immersive, human-in-the-loop multi-robot navigation.

Problem

• multi-robot systems
• unknown environments
• human-in-the-loop
• shared control
• operator intent

Innovation

• shared control
• motion primitive
• virtual reality
• multi-robot navigation
• admittance control