coordinate frame alignment

This study investigates how to effectively organize heterogeneous robotic demonstration data to enhance cross-embodiment transfer performance. Through controlled simulation experiments, it systematically compares the efficacy of unpaired large-scale data against structured paired data—such as demonstrations aligned by scene, task, or trajectory—under varying morphologies and viewpoints. The findings reveal that, for morphology differences, structured data analogies are more effective than merely increasing data diversity, highlighting distinct data structure requirements for morphology transfer versus viewpoint transfer. By optimizing data composition alone, the approach achieves an average 22.5% improvement in success rate on real-world cross-embodiment transfer tasks, underscoring the critical role of data analogy in enabling effective embodiment-agnostic skill transfer.

This work addresses the challenge of policy transfer across robotic morphologies, which is hindered by embodiment differences that limit the generalization of imitation learning. To overcome this, the authors propose leveraging embodiment-invariant behavioral alignment representations—such as end-effector trajectories, object bounding boxes, and language-based action descriptions—to construct a unified vision-language-action (VLA) model capable of integrating multi-embodiment data and enabling effective cross-embodiment transfer. The approach presents the first systematic evaluation of behavioral alignment representations in sim-to-real transfer, demonstrating substantial performance gains on a newly introduced simulation benchmark. When deployed on real robots, the method improves task success rates by 28% and further enables training augmentation using unlabeled demonstration data lacking action annotations.

Accurate modeling of fine-grained spatial relationships among objects—such as co-occurrence of visually similar objects and multi-level spatial nesting—is critical for real-world robotic task planning in complex indoor environments; however, existing datasets lack 3D positional annotations and explicit spatial relation labels. To address this gap, we introduce SpotSRD, the first spatial-relation-aware dataset designed for real-robot deployment, comprising nearly 1,000 indoor images captured by Boston Dynamics’ Spot robot, with fine-grained annotations of object attributes, 3D coordinates, and 24 spatial relation types. We develop a custom annotation tool and a scene-graph evaluation framework, systematically exposing spatial modeling bottlenecks across six state-of-the-art models. Furthermore, we propose a spatially aware prompting mechanism that injects structured spatial relations into GPT-4o, significantly improving its accuracy on spatial planning tasks. The dataset, annotation tool, and code are publicly released.

Existing tactile simulators struggle to accurately replicate the complex deformations and transduction mechanisms of real sensors, limiting sim-to-real transfer performance. This work proposes a multimodal representation learning framework that maps heterogeneous tactile signals—such as simulated penetration depth and real capacitive readings—into a shared latent space using modality-specific encoders. The model is trained with self-reconstruction, cross-reconstruction, and contrastive alignment losses, enabling zero-shot transfer without requiring high-fidelity simulation of raw sensory signals. By integrating multiphysics simulation to enrich embedding informativeness and leveraging a Warp-accelerated penalty-based contact model for computational efficiency, the approach achieves a 16.7% reduction in force prediction error and a 45.8% decrease in shape reconstruction error. It further demonstrates successful zero-shot cross-modal transfer across multiple downstream tasks and includes an open-sourced, efficient tactile simulation module.

This work addresses two core challenges in large-scale robotic manipulation datasets: (1) designing high-value diversity dimensions to enhance data utility, and (2) efficiently retrieving task-aligned demonstrations from existing datasets. To this end, we introduce a programmable data generation framework that explicitly models controllable diversity variables—including camera pose, object categories, and spatial layout. Our analysis reveals, for the first time, that camera pose and spatial arrangement are critical determinants of both dataset diversity and task alignment. We further propose a task-oriented demonstration retrieval algorithm grounded in geometric-semantic joint alignment. Evaluated on real-world datasets including DROID, our method improves downstream policy performance by up to 70%. Crucially, insights and gains observed in simulation generalize successfully to physical robot platforms, demonstrating robust cross-domain transferability.

This work addresses the limitation of existing robotic vision policies that tightly couple scene geometry with specific viewpoints, hindering effective learning from multi-view data and generalization to novel perspectives. To overcome this, the authors propose ARGUS, the first approach to integrate large-scale 3D vision models into robotic observation preprocessing. By aligning images from arbitrary viewpoints into a canonical coordinate frame, ARGUS decouples scene geometry from viewpoint dependency, substantially reducing the learning burden on downstream visuomotor policies. Evaluated on both fixed and highly diverse viewpoint datasets, the method consistently outperforms current state-of-the-art approaches, achieving 4–6× faster training convergence and significantly higher task success rates.

This work addresses the limited cross-platform transferability of existing tactile perception strategies, which are heavily dependent on specific sensor modalities. To overcome this, the study proposes a unified, sensor-agnostic tactile representation by constructing a shared latent space across three heterogeneous tactile modalities—resistive, magnetic, and vision-based. This is achieved through modality-specific encoders, pairwise contact alignment signals, and joint training. The resulting representation enables zero-shot transfer of tactile policies across sensor types. Evaluated on four contact-intensive manipulation tasks, the method significantly improves average success rates from 27.5% to 45.9%, demonstrating effective disentanglement and generalization of cross-modal tactile perception and manipulation.

This work investigates whether existing action-conditioned world models can generalize to unseen robot morphologies beyond mere visual memorization. To this end, the authors introduce XEWorld, a cross-embodiment evaluation benchmark that establishes, for the first time, an isolated-embodiment assessment paradigm. This framework systematically evaluates zero-shot and few-shot visual rendering capabilities of models when confronted with novel robots that share physical consistency but differ in embodiment structure. Experiments reveal that current models struggle to map abstract joint actions into coherent visual trajectories, relying heavily on visual similarity rather than kinematic or dynamic consistency for generalization. Furthermore, few-shot adaptation often leads to catastrophic forgetting of previously seen embodiments. These findings underscore the critical need for architectural innovations that explicitly disentangle appearance from physical dynamics.

This work addresses the lack of a systematic organizational framework for embodied intelligence data, which hinders scalability and robot alignment. The authors propose a “Data Pyramid” framework that introduces a hierarchical structure to integrate five heterogeneous data sources—real robot data, UMI-style datasets, first- and third-person videos, simulation data, and general vision-language corpora—organized according to quality, diversity, reusability, and physical fidelity. Through multimodal data evaluation, alignment strategies, and hybrid pretraining, the study systematically analyzes how data composition influences model perception, reasoning, and planning capabilities. The work establishes design principles for data selection and combination in embodied foundation models and identifies six key open challenges, thereby advancing the development of data infrastructure for embodied learning.

Existing vision-language-action (VLA) models primarily focus on semantic alignment, often falling short in capturing the geometric awareness and dynamic manipulability required for embodied tasks. This work proposes GeoAlign, a novel architecture that introduces, for the first time, an ego-state-guided geometric feature querying mechanism. Specifically, the RGB branch is post-trained under RGB-D supervision to generate geometry-enhanced features, which are then dynamically queried using the robot’s ego-state to extract phase-relevant geometric tokens for action prediction. By preserving semantic understanding while achieving spatially precise alignment, GeoAlign substantially improves policy generalization on complex geometric tasks, attaining 99.0% success on LIBERO, an average of 85.3% across three SimplerEnv-Fractal tasks, and 78.8% success rate on eight real-world ALOHA geometric manipulation tasks.