SAP-CoPE: Social-Aware Planning using Cooperative Pose Estimation with Infrastructure Sensor Nodes

2025-04-08

AI Summary

In indoor human-robot coexistence scenarios, pure onboard perception suffers from inaccurate human intention recognition and socially uncomfortable trajectory planning. To address this, we propose a real-time planning framework integrating roadside cooperative perception with social norm constraints. Methodologically, we pioneer the incorporation of a personal space field—derived from human pose modeling—into model predictive control (MPC); design a 3D pose estimation algorithm that jointly accounts for projection uncertainty and joint-level geometric consistency; and support multi-source fusion across single/multi-camera and sparse LiDAR inputs. Experiments in both simulated and real-world indoor environments demonstrate a 23.6% improvement in human intention recognition accuracy, a planning success rate exceeding 94.1%, and significant enhancements in trajectory social comfort and socially compliant avoidance behavior.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Social recommender systems and personalization for the social Web
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Autonomous driving systems must operate safely in human-populated indoor environments, where challenges such as limited perception and occlusion sensitivity arise when relying solely on onboard sensors. These factors generate difficulties in the accurate recognition of human intentions and the generation of comfortable, socially aware trajectories. To address these issues, we propose SAP-CoPE, a social-aware planning framework that integrates cooperative infrastructure with a novel 3D human pose estimation method and a model predictive control-based controller. This real-time framework formulates an optimization problem that accounts for uncertainty propagation in the camera projection matrix while ensuring human joint coherence. The proposed method is adaptable to single- or multi-camera configurations and can incorporate sparse LiDAR point-cloud data. To enhance safety and comfort in human environments, we integrate a human personal space field based on human pose into a model predictive controller, enabling the system to navigate while avoiding discomfort zones. Extensive evaluations in both simulated and real-world settings demonstrate the effectiveness of our approach in generating socially aware trajectories for autonomous systems.

Problem

• Improving autonomous driving safety in human-populated indoor environments
• Enhancing human intention recognition and trajectory planning accuracy
• Integrating cooperative infrastructure for real-time social-aware navigation

Innovation

• Integrates cooperative infrastructure with 3D pose estimation
• Uses model predictive control for human comfort
• Adapts to single or multi-camera configurations