PEDESTRIAN: An Egocentric Vision Dataset for Obstacle Detection on Pavements

2025-12-22

AI Summary

Urban sidewalks are frequently obstructed by hazards that compromise pedestrian safety, yet real-time detection is hindered by the absence of high-quality, multi-class egocentric visual datasets. To address this gap, we introduce the first large-scale egocentric video dataset specifically designed for sidewalk obstacle detection—comprising 340 real-world smartphone-recorded videos spanning 29 common obstacle categories. We systematically define and publicly release a high-fidelity, fine-grained annotation benchmark, the first of its kind, thereby filling a critical void in open pedestrian safety resources. Leveraging this dataset, we conduct a comprehensive evaluation of state-of-the-art object detectors—including YOLOv8 and Mask R-CNN—establishing fully reproducible baselines. Our best-performing model achieves a mean average precision (mAP@0.5) of 68.3%. This work provides both an essential data foundation and an authoritative performance benchmark for developing robust pedestrian safety warning systems.

Technology Category

• Computer Vision: Object Detection & Categorization
• Natural Language Processing: Safety and Robustness
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Walking has always been a primary mode of transportation and is recognized as an essential activity for maintaining good health. Despite the need for safe walking conditions in urban environments, sidewalks are frequently obstructed by various obstacles that hinder free pedestrian movement. Any object obstructing a pedestrian's path can pose a safety hazard. The advancement of pervasive computing and egocentric vision techniques offers the potential to design systems that can automatically detect such obstacles in real time, thereby enhancing pedestrian safety. The development of effective and efficient identification algorithms relies on the availability of comprehensive and well-balanced datasets of egocentric data. In this work, we introduce the PEDESTRIAN dataset, comprising egocentric data for 29 different obstacles commonly found on urban sidewalks. A total of 340 videos were collected using mobile phone cameras, capturing a pedestrian's point of view. Additionally, we present the results of a series of experiments that involved training several state-of-the-art deep learning algorithms using the proposed dataset, which can be used as a benchmark for obstacle detection and recognition tasks. The dataset can be used for training pavement obstacle detectors to enhance the safety of pedestrians in urban areas.

Problem

• Detects sidewalk obstacles for pedestrian safety
• Introduces egocentric vision dataset for urban environments
• Trains deep learning models for obstacle recognition

Innovation

• Introduces PEDESTRIAN dataset with egocentric videos
• Trains deep learning models for obstacle detection
• Provides benchmark for real-time pedestrian safety enhancement