LiHRA: A LiDAR-Based HRI Dataset for Automated Risk Monitoring Methods

2025-09-08

AI Summary

Existing HRI datasets suffer from insufficient real-world hazardous scenarios and lack of multimodal, high-precision temporal synchronization—hindering the development of robust risk monitoring algorithms. To address this, we introduce the first multimodal risk monitoring dataset specifically designed for industrial collaborative robotics. It uniquely integrates synchronized 3D LiDAR point clouds, human skeletal keypoints, and robot joint states across six representative interaction scenarios—spanning both safe and hazardous human–robot engagements. Leveraging LiDAR acquisition, cross-modal spatiotemporal alignment, and dynamic modeling, we achieve millisecond-level (10 Hz) synchronization of human–robot–environment interactions and release 4,431 frames of finely annotated point clouds. We further propose a context-aware risk quantification method, empirically validated on a unified benchmark to ensure consistency between AI-driven and conventional risk assessment approaches. This work fills a critical gap in high-quality, time-resolved, hazard-aware HRI data.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We present LiHRA, a novel dataset designed to facilitate the development of automated, learning-based, or classical risk monitoring (RM) methods for Human-Robot Interaction (HRI) scenarios. The growing prevalence of collaborative robots in industrial environments has increased the need for reliable safety systems. However, the lack of high-quality datasets that capture realistic human-robot interactions, including potentially dangerous events, slows development. LiHRA addresses this challenge by providing a comprehensive, multi-modal dataset combining 3D LiDAR point clouds, human body keypoints, and robot joint states, capturing the complete spatial and dynamic context of human-robot collaboration. This combination of modalities allows for precise tracking of human movement, robot actions, and environmental conditions, enabling accurate RM during collaborative tasks. The LiHRA dataset covers six representative HRI scenarios involving collaborative and coexistent tasks, object handovers, and surface polishing, with safe and hazardous versions of each scenario. In total, the data set includes 4,431 labeled point clouds recorded at 10 Hz, providing a rich resource for training and benchmarking classical and AI-driven RM algorithms. Finally, to demonstrate LiHRA's utility, we introduce an RM method that quantifies the risk level in each scenario over time. This method leverages contextual information, including robot states and the dynamic model of the robot. With its combination of high-resolution LiDAR data, precise human tracking, robot state data, and realistic collision events, LiHRA offers an essential foundation for future research into real-time RM and adaptive safety strategies in human-robot workspaces.

Problem

• Lack of high-quality datasets for realistic human-robot interaction scenarios
• Need for reliable safety systems in collaborative industrial robot environments
• Difficulty in developing accurate risk monitoring methods without comprehensive multimodal data

Innovation

• LiDAR-based dataset for human-robot interaction monitoring
• Combines 3D point clouds with human keypoints and robot states
• Enables real-time risk quantification using contextual information