PMOF: A Dataset and Benchmark for Passenger Monitoring Using Overhead Fisheye Cameras

2026-06-11

AI Summary

This work addresses the challenges of overhead fisheye-view passenger monitoring in autonomous buses, where limited space, dynamic lighting, moving backgrounds, and occlusions hinder performance. Existing datasets, collected only in static environments, lack generalizability to real-world mobile scenarios. To bridge this gap, the authors introduce PMOF—the first overhead fisheye dataset specifically designed for moving vehicle contexts—comprising over 19,000 manually annotated frames supporting object detection, tracking, and action recognition. They establish a multitask benchmark by incorporating rotated bounding boxes, tracking IDs, and action labels. Leveraging cross-domain fine-tuning and rotation-aware data augmentation, their approach achieves 94.8% AP50 on PMOF and demonstrates strong generalization with 96.5% AP50 on unseen-domain datasets, significantly advancing model robustness in dynamic transit environments.

Abstract

Autonomous staff-free public transport requires reliable in-vehicle passenger monitoring. However, perception inside moving vehicles is challenged by confined spaces, variable illumination, motion-induced background variation, occlusion, and limited viewpoints. To mitigate these spatial constraints, ceiling-mounted fisheye cameras provide full-scene coverage from a single viewpoint. Yet existing public overhead fisheye datasets are recorded in static environments and do not capture the domain shift introduced by vehicle motion. To fill this gap, we introduce PMOF, Passenger Monitoring using Overhead Fisheye cameras, the first public dataset of top-view fisheye imagery captured inside a moving vehicle, comprising over 19k manually annotated frames. PMOF provides rotated bounding boxes, tracking identifiers, and action labels, supporting object detection, tracking, and action recognition. We benchmark PMOF using YOLO26m-obb models fine-tuned under multiple dataset configurations that combine PMOF with existing overhead fisheye datasets. Cross-domain fine-tuning with custom rotation-aware augmentation achieves 94.8% AP50 on PMOF and 96.5% AP50 on an unseen overhead fisheye dataset from a different domain. Our results highlight the domain gap between static and moving environments and show that incorporating PMOF improves detection performance and advances generalization beyond passenger monitoring to broader fisheye-based person detection tasks. The dataset and code are available at https://swermuth.github.io/pmof/.

Problem

• passenger monitoring
• overhead fisheye cameras
• domain shift
• moving vehicle
• in-vehicle perception

Innovation

• overhead fisheye cameras
• moving vehicle dataset
• rotated bounding boxes
• cross-domain fine-tuning
• rotation-aware augmentation