CHOICE: Coordinated Human-Object Interaction in Cluttered Environments for Pick-and-Place Actions

2024-12-09

arXiv.org

AI Summary

To address the challenges of poor generalizability and unnatural transitions in human-object collaborative pick-and-place animation generation within cluttered environments, this paper proposes a hierarchical goal-driven framework. First, a bimanual scheduler generates task-critical keyframes; second, a neural implicit planner models hand trajectories adaptively to diverse object geometries and dynamic obstacle configurations; third, a DeepPhase controller—enhanced by Kalman-filter-augmented frequency-domain smoothing—enables multi-target linear dynamical motion control. Our key innovations include the first integration of bimanual coordination with neural implicit planning, and a novel frequency-domain-optimized DeepPhase dynamic control paradigm. Experiments demonstrate significant improvements in task success rate and motion naturalness across photorealistic scenarios featuring geometric heterogeneity, movable containers, and dense layouts, outperforming existing single-arm and static-planning approaches.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Animating human-scene interactions such as pick-and-place tasks in cluttered, complex layouts is a challenging task, with objects of a wide variation of geometries and articulation under scenarios with various obstacles. The main difficulty lies in the sparsity of the motion data compared to the wide variation of the objects and environments as well as the poor availability of transition motions between different tasks, increasing the complexity of the generalization to arbitrary conditions. To cope with this issue, we develop a system that tackles the interaction synthesis problem as a hierarchical goal-driven task. Firstly, we develop a bimanual scheduler that plans a set of keyframes for simultaneously controlling the two hands to efficiently achieve the pick-and-place task from an abstract goal signal such as the target object selected by the user. Next, we develop a neural implicit planner that generates guidance hand trajectories under diverse object shape/types and obstacle layouts. Finally, we propose a linear dynamic model for our DeepPhase controller that incorporates a Kalman filter to enable smooth transitions in the frequency domain, resulting in a more realistic and effective multi-objective control of the character.Our system can produce a wide range of natural pick-and-place movements with respect to the geometry of objects, the articulation of containers and the layout of the objects in the scene.

Problem

• Generating human motions for pick-and-place in cluttered environments
• Overcoming sparse motion data and poor transition availability
• Synthesizing interactions under diverse object geometries and obstacles

Innovation

• Hierarchical goal-driven task for interaction synthesis
• Neural implicit planner for hand trajectory generation
• Linear dynamic model with Kalman filter control