Scene-aware Prediction of Diverse Human Movement Goals

2026-06-29

AI Summary

This work addresses the challenge of predicting diverse, goal-driven human motion trajectories in complex environments without relying on explicit scene semantic annotations. To this end, the authors propose a novel conditional variational autoencoder (CVAE) framework that jointly encodes RGB scene images and human poses, enabling sampling in the latent space to generate multiple plausible motion goals consistent with environmental context. By leveraging this joint representation, the method models both the diversity of human goal-oriented behaviors and their alignment with scene constraints, all without requiring explicit semantic supervision. Experimental validation on the GTA-IM and PROX datasets demonstrates the approach’s effectiveness and strong generalization across diverse scenes.

Abstract

Anticipation of human behaviours facilitates autonomous systems in proactive planning. Human behaviour could be stochastic due to varying goals. Human goals typically guide their own movement and could therefore help to predict the human trajectory and human motion in the long-term. To infer the human movement intentions, the environmental context plays a significant role, in addition to the social cues expressed by the individual. Previous works on human goals prediction either require semantic knowledge of the scene, or only tackle interactions with objects. In this paper, we propose a novel multi-goal prediction method using the generative model to address the stochasticity of human movement. It leverages the current RGB scene and the human pose to predict diverse potential future goals of human movement based on the Conditional Variational Autoencoder (CVAE). Our results demonstrate that our approach is capable of generating multiple movement goals in the scene via samplings in latent space of the CVAE and exhibits generalization capability across scenarios in GTA-IM dataset and PROX dataset. Code is publicly available at \href{https://github.com/Q-Y-Yang/DiverseGoalsPrediction.git}{\texttt{https://github.com/Q-Y-Yang/DiverseGoalsPrediction}}.

Problem

• human movement prediction
• goal anticipation
• scene-aware
• stochastic behavior
• diverse goals

Innovation

• multi-goal prediction
• Conditional Variational Autoencoder
• scene-aware
• human motion anticipation
• diverse trajectory generation