Monocular Person Localization under Camera Ego-motion

2025-03-04

AI Summary

Existing monocular human 3D localization methods for mobile platforms suffer from severe degradation under aggressive ego-motion and often rely on restrictive static-camera assumptions or exhibit poor generalization. To address this, we propose a joint optimization framework that reformulates localization as pose estimation. Leveraging a minimal four-point human geometric model, our method simultaneously optimizes the camera’s 2D pose and the subject’s 3D position, integrating monocular geometric constraints while abandoning both conventional regression-based paradigms and the static-camera assumption. The approach enables real-time embedded deployment and achieves substantial accuracy improvements—reducing average localization error by 37.2%—on public benchmarks and real-world quadruped robot experiments. It further enables a robust autonomous following system. Our core contribution is the first end-to-end, nonlinear joint optimization of monocular human localization and camera motion estimation, establishing a novel paradigm for human–robot interaction (HRI) on dynamic platforms.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Localizing a person from a moving monocular camera is critical for Human-Robot Interaction (HRI). To estimate the 3D human position from a 2D image, existing methods either depend on the geometric assumption of a fixed camera or use a position regression model trained on datasets containing little camera ego-motion. These methods are vulnerable to fierce camera ego-motion, resulting in inaccurate person localization. We consider person localization as a part of a pose estimation problem. By representing a human with a four-point model, our method jointly estimates the 2D camera attitude and the person's 3D location through optimization. Evaluations on both public datasets and real robot experiments demonstrate our method outperforms baselines in person localization accuracy. Our method is further implemented into a person-following system and deployed on an agile quadruped robot.

Problem

• Localizing a person from a moving monocular camera.
• Estimating 3D human position from 2D images with camera motion.
• Improving accuracy in person localization under camera ego-motion.

Innovation

• Uses four-point model for human representation
• Jointly estimates camera attitude and 3D location
• Implemented in agile quadruped robot system