ARCAS: An Augmented Reality Collision Avoidance System with SLAM-Based Tracking for Enhancing VRU Safety

2025-12-04

AI Summary

Vulnerable road users (VRUs) face high collision risks in mixed-traffic environments, yet existing safety systems lack direct, real-time support for them. Method: This work proposes the first LiDAR-driven, real-time augmented reality (AR) collision warning system specifically designed for VRUs. It integrates roadside 360° 3D LiDAR perception, head-mounted display (HMD)-based SLAM pose tracking, and automatic 3D calibration to render georeferenced dynamic 3D bounding boxes and directional arrows within the user’s AR field of view. A novel multi-HMD shared spatial anchor coordination mechanism ensures cross-device AR spatial consistency. Results: Evaluated across 180 real-world traffic interactions, the system increased time-to-collision for pedestrians by an average of 97% and improved third-party reaction margins by up to 4× compared to unassisted (naked-eye) conditions—providing the first empirical evidence of significant safety gains enabled by LiDAR-AR guidance for VRUs.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Vulnerable road users (VRUs) face high collision risks in mixed traffic, yet most existing safety systems prioritize driver or vehicle assistance over direct VRU support. This paper presents ARCAS, a real-time augmented reality collision avoidance system that provides personalized spatial alerts to VRUs via wearable AR headsets. By fusing roadside 360-degree 3D LiDAR with SLAM-based headset tracking and an automatic 3D calibration procedure, ARCAS accurately overlays world-locked 3D bounding boxes and directional arrows onto approaching hazards in the user's passthrough view. The system also enables multi-headset coordination through shared world anchoring. Evaluated in real-world pedestrian interactions with e-scooters and vehicles (180 trials), ARCAS nearly doubled pedestrians' time-to-collision and increased counterparts' reaction margins by up to 4x compared to unaided-eye conditions. Results validate the feasibility and effectiveness of LiDAR-driven AR guidance and highlight the potential of wearable AR as a promising next-generation safety tool for urban mobility.

Problem

• Develops AR system for VRU safety with real-time alerts
• Uses LiDAR and SLAM for accurate hazard visualization
• Enhances collision avoidance in mixed traffic via AR guidance

Innovation

• SLAM-based AR headset tracking for real-time hazard overlay
• Fusion of roadside LiDAR with wearable AR for spatial alerts
• Multi-headset coordination via shared world anchoring system