Generating Continual Human Motion in Diverse 3D Scenes

2023-04-04

International Conference on 3D Vision

AI Summary

Generating long-horizon, multi-action coherent human motions in 3D scenes remains challenging due to drift accumulation, action discontinuity, and poor scene adaptability. Method: We propose an animator-guided, scene-agnostic iterative generation framework. It establishes a target-centric canonical coordinate system to decouple path planning from motion transition, and employs motion decomposition modeling with coordinate-system reparameterization—enabling zero-shot deployment on pure motion-capture data without scene-aware annotations or fine-tuning. Contribution/Results: To our knowledge, this is the first method to generate drift-free, chained multi-action sequences (e.g., “grasp → sit → lean”) in diverse real-world scanned environments—including HPS, Replica, Matterport, and ScanNet—using only sparse keypoint constraints and a seed motion. Unlike existing 3D navigation approaches, ours requires no scene rendering, geometric encoding, or environment-specific training, achieving superior generalization, motion plausibility, and scene compatibility.

Abstract

We introduce a method to synthesize animator guided human motion across 3D scenes. Given a set of sparse (3 or 4) joint locations (such as the location of a person’s hand and two feet) and a seed motion sequence in a 3D scene, our method generates a plausible motion sequence starting from the seed motion while satisfying the constraints imposed by the provided keypoints. We decompose the continual motion synthesis problem into walking along paths and transitioning in and out of the actions specified by the keypoints, which enables long generation of motions that satisfy scene constraints without explicitly incorporating scene information. Our method is trained only using scene agnostic mocap data. As a result, our approach is deployable across 3D scenes with various geometries. For achieving plausible continual motion synthesis without drift, our key contribution is to iteratively generate motion in a goal-centric canonical coordinate frame where the next immediate target is situated at the origin. Our model can generate long sequences of diverse actions such as grabbing, sitting and leaning chained together in arbitrary order, demonstrated on scenes of varying geometry: HPS, Replica, Matterport, ScanNet scenes. Several experiments demonstrate that our method outperforms existing methods that navigate paths in 3D scenes.

Problem

• 3D Human Motion Simulation
• Coherence and Naturalness
• Adaptability to Different 3D Environments

Innovation

• 3D Human Motion Generation
• Scene-agnostic Learning
• Goal-centric Coordinate System