Sixth-Sense: Self-Supervised Learning of Spatial Awareness of Humans from a Planar Lidar

2025-02-28

AI Summary

Service robots operating in close proximity to humans require low-cost, omnidirectional human perception. Method: We propose the first self-supervised method for human detection and 2D pose estimation using only a single-line (1D) LiDAR. Cross-modal knowledge distillation from RGB-D detection transfers narrow-field supervision to omnidirectional 1D point sequences without manual annotation; a lightweight CNN-LSTM architecture models temporal point sequences and enables environment-adaptive training. Contribution/Results: Our approach achieves 71% precision and 80% recall in novel scenes, with mean errors of 13 cm in distance and 44° in orientation—trained autonomously on just 70 minutes of collected data. This work is the first to demonstrate that a single-line LiDAR, under self-supervision, can support robust spatial human perception—substantially reducing hardware cost and enhancing deployment generalizability.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Localizing humans is a key prerequisite for any service robot operating in proximity to people. In these scenarios, robots rely on a multitude of state-of-the-art detectors usually designed to operate with RGB-D cameras or expensive 3D LiDARs. However, most commercially available service robots are equipped with cameras with a narrow field of view, making them blind when a user is approaching from other directions, or inexpensive 1D LiDARs whose readings are difficult to interpret. To address these limitations, we propose a self-supervised approach to detect humans and estimate their 2D pose from 1D LiDAR data, using detections from an RGB-D camera as a supervision source. Our approach aims to provide service robots with spatial awareness of nearby humans. After training on 70 minutes of data autonomously collected in two environments, our model is capable of detecting humans omnidirectionally from 1D LiDAR data in a novel environment, with 71% precision and 80% recall, while retaining an average absolute error of 13 cm in distance and 44{deg} in orientation.

Problem

• Localizing humans for service robots using 1D LiDAR data.
• Overcoming limitations of narrow field-of-view cameras and expensive sensors.
• Enabling omnidirectional human detection and pose estimation in novel environments.

Innovation

• Self-supervised learning for human detection
• Uses 1D LiDAR data with RGB-D supervision
• Omnidirectional human pose estimation from LiDAR