Real-time Recognition of Human Interactions from a Single RGB-D Camera for Socially-Aware Robot Navigation

2025-09-29

AI Summary

Existing robot navigation systems often neglect human social cues, resulting in unnatural interactions and compromised safety. This paper proposes a real-time, socially aware navigation method based on a single RGB-D camera. First, 3D human pose estimation extracts individual body configurations; then, a computationally efficient spatial group modeling approach—integrating Principal Component Analysis (PCA) with the shoelace formula—precisely identifies interaction directions and participant regions. The entire pipeline is implemented end-to-end within the ROS 2 framework. Evaluated in dynamic social environments, the method achieves high-accuracy social interaction recognition with an average per-frame processing time of approximately 4 ms, enabling deployment on resource-constrained embedded platforms. The source code is publicly released. This work establishes a novel paradigm for lightweight, scalable, and socially compliant robotic navigation.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: Social recommender systems and personalization for the social Web
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

{Recognizing human interactions is essential for social robots as it enables them to navigate safely and naturally in shared environments. Conventional robotic systems however often focus on obstacle avoidance, neglecting social cues necessary for seamless human-robot interaction. To address this gap, we propose a framework to recognize human group interactions for socially aware navigation. Our method utilizes color and depth frames from a monocular RGB-D camera to estimate 3D human keypoints and positions. Principal component analysis (PCA) is then used to determine dominant interaction directions. The shoelace formula is finally applied to compute interest points and engagement areas. Extensive experiments have been conducted to evaluate the validity of the proposed method. The results show that our method is capable of recognizing group interactions across different scenarios with varying numbers of individuals. It also achieves high-speed performance, processing each frame in approximately 4 ms on a single-board computer used in robotic systems. The method is implemented as a ROS 2 package making it simple to integrate into existing navigation systems. Source code is available at https://github.com/thanhlong103/social-interaction-detector

Problem

• Recognizing human group interactions for socially-aware robot navigation
• Estimating 3D human keypoints from single RGB-D camera input
• Determining interaction directions and engagement areas computationally

Innovation

• RGB-D camera estimates 3D human keypoints and positions
• PCA determines dominant interaction directions
• Shoelace formula computes engagement areas and interest points