DataLadder: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid

2026-06-15

AI Summary

This work addresses the lack of scalable, trustworthy evaluation methods and physically plausible training data for general-purpose robotic policies, compounded by the high cost and poor reproducibility of real-world robot experiments. To overcome these challenges, the authors propose a human↔simulation↔robot bidirectional alignment framework supported by a cloud-native toolchain. Leveraging the JoySim simulator—integrated with reconstruction, rendering, and realism-enhancement modules—they implement a high-fidelity digital twin on the JD Cloud platform. Human demonstrations are transformed into physically consistent trajectories, annotations, and visual observations, while simulation serves dual roles as a scalable evaluation layer and a data filter. This approach substantially improves both the efficiency of data generation and the reliability of policy evaluation.

Abstract

Generalist robot policies require trustworthy evaluation and robot-usable training data, but both are difficult to scale with physical robots alone. Real-robot trials and demonstrations remain the most faithful source of deployment signals, yet they are slow, costly, and hard to reproduce. We present DataLadder, a simulation-enabled interconversion toolchain for human-robot aligned model evaluation and data generation, denoted as Robot $\rightleftharpoons$ Simulation $\rightleftharpoons$ Human. On the one hand, the Robot $\rightarrow$ Simulation $\rightarrow$ Human pathway supports human-robot aligned model evaluation by reconstructing real-robot tabletop organization tasks as calibrated digital twins for scalable evaluation, while using human embodied feedback to inspect and refine the naturalness of simulated motions. On the other hand, the Human $\rightarrow$ Simulation $\rightarrow$ Robot pathway supports human-robot aligned data generation: it lifts ego-centric human demonstrations into simulation, checks them under robot physical constraints, and converts them into robot-centered trajectories, annotations, and visual observations. Together, these pathways use the JoySim simulator as both a scalable evaluation layer and a physical consistency filter for robot data generation. We further package the core reconstruction, simulation, rendering, and realism-augmentation modules as cloud services on JD Cloud, turning the system into reusable infrastructure for robot data generation and model evaluation.

Problem

• robot evaluation
• training data scalability
• human-robot alignment
• simulation fidelity
• embodied data

Innovation

• simulation-enabled interconversion
• embodied data pyramid
• digital twin
• human-robot alignment
• cloud-based robot data infrastructure