Towards Motion Turing Test: Evaluating Human-Likeness in Humanoid Robots

2026-03-06

AI Summary

This work proposes the "Action Turing Test" framework to objectively evaluate the human-likeness of humanoid robot motions—specifically, whether human observers can distinguish the source of an action based solely on kinematic information. To facilitate this assessment, the authors introduce HHMotion, the first large-scale, standardized dataset comprising 1,000 motion sequences from both humans and robots, all unified in the SMPL-X representation, along with human-likeness ratings provided by 30 annotators. Experimental results demonstrate that current large language models exhibit limited performance on this task, whereas the proposed lightweight baseline model achieves significantly better results. The dataset, code, and benchmark will be publicly released to advance research in humanoid robot motion evaluation.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Humanoid robots have achieved significant progress in motion generation and control, exhibiting movements that appear increasingly natural and human-like. Inspired by the Turing Test, we propose the Motion Turing Test, a framework that evaluates whether human observers can discriminate between humanoid robot and human poses using only kinematic information. To facilitate this evaluation, we present the Human-Humanoid Motion (HHMotion) dataset, which consists of 1,000 motion sequences spanning 15 action categories, performed by 11 humanoid models and 10 human subjects. All motion sequences are converted into SMPL-X representations to eliminate the influence of visual appearance. We recruited 30 annotators to rate the human-likeness of each pose on a 0-5 scale, resulting in over 500 hours of annotation. Analysis of the collected data reveals that humanoid motions still exhibit noticeable deviations from human movements, particularly in dynamic actions such as jumping, boxing, and running. Building on HHMotion, we formulate a human-likeness evaluation task that aims to automatically predict human-likeness scores from motion data. Despite recent progress in multimodal large language models, we find that they remain inadequate for assessing motion human-likeness. To address this, we propose a simple baseline model and demonstrate that it outperforms several contemporary LLM-based methods. The dataset, code, and benchmark will be publicly released to support future research in the community.

Problem

• Motion Turing Test
• human-likeness
• humanoid robots
• motion evaluation
• kinematic information

Innovation

• Motion Turing Test
• Humanoid Motion Evaluation
• HHMotion Dataset
• SMPL-X Representation
• Human-Likeness Prediction