A Vehicle System for Navigating Among Vulnerable Road Users Including Remote Operation

2025-05-08

AI Summary

To address autonomous driving safety challenges in scenarios with high densities of vulnerable road users (VRUs) and edge cases such as construction zones and emergency response, this paper proposes a dual-mode vehicle system integrating autonomous navigation and remote takeover. Methodologically: (1) we design a topology-driven model predictive control (T-MPC) motion planner that generates parallel multi-strategy trajectories under joint probabilistic VRU collision constraints to enhance safety; (2) we introduce a novel vision–haptics fusion teleoperation interface to improve human–machine collaboration in edge scenarios; and (3) we integrate multi-sensor fusion perception, robust SLAM-based mapping, and closed-loop vehicle control. Experimental results demonstrate that the system achieves significantly higher safety and traffic efficiency than three baseline approaches in simulation; real-world closed-course tests successfully validate stable operation in both autonomous and remote-controlled modes.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Security and Privacy: Large-scale security measurements

Abstract

We present a vehicle system capable of navigating safely and efficiently around Vulnerable Road Users (VRUs), such as pedestrians and cyclists. The system comprises key modules for environment perception, localization and mapping, motion planning, and control, integrated into a prototype vehicle. A key innovation is a motion planner based on Topology-driven Model Predictive Control (T-MPC). The guidance layer generates multiple trajectories in parallel, each representing a distinct strategy for obstacle avoidance or non-passing. The underlying trajectory optimization constrains the joint probability of collision with VRUs under generic uncertainties. To address extraordinary situations ("edge cases") that go beyond the autonomous capabilities - such as construction zones or encounters with emergency responders - the system includes an option for remote human operation, supported by visual and haptic guidance. In simulation, our motion planner outperforms three baseline approaches in terms of safety and efficiency. We also demonstrate the full system in prototype vehicle tests on a closed track, both in autonomous and remotely operated modes.

Problem

• Developing a vehicle system for safe navigation around Vulnerable Road Users (VRUs)
• Introducing T-MPC motion planner for optimized obstacle avoidance strategies
• Incorporating remote human operation for handling extraordinary edge cases

Innovation

• Topology-driven Model Predictive Control (T-MPC) planner
• Parallel trajectory generation for obstacle avoidance
• Remote human operation with visual and haptic guidance