mujoco

Existing embodied reasoning agents exhibit limited performance on long-horizon manipulation tasks requiring physical interaction for information acquisition (e.g., “sort objects by weight”) and suffer from poor generalization beyond their training environments. Method: We introduce the first high-fidelity simulation platform for long-horizon robotic manipulation, integrating MuJoCo’s accurate physics modeling with Blender’s photorealistic rendering to enable dual closed-loop interaction—vision-action and control-physics. Contribution/Results: Our platform is the first to jointly achieve high physical fidelity and visual realism for long-sequence tasks. We propose SHOP-VRB2, a benchmark comprising 10 multi-step reasoning scenarios that demand coordinated visual and physical measurements. Built upon robosuite, the platform is open-sourced to support multimodal data generation and closed-loop policy training. Experiments demonstrate substantial improvements in agents’ planning and reasoning capabilities on complex manipulation tasks, establishing critical infrastructure for long-horizon embodied reasoning systems.

Existing physics engines for robotics struggle to simultaneously ensure stable simulation, high-fidelity rigid contact modeling, and full differentiability with respect to states, actions, and system parameters. To address this, we propose Dojo—the first end-to-end differentiable physics engine designed specifically for robotics. Dojo uniquely integrates variational integrators with a second-order cone nonlinear complementarity problem (NCP) solver, guaranteeing energy and momentum conservation during contact and enabling smooth, analytic gradient computation across contact events. It further employs a customized primal-dual interior-point method for efficient implicit differentiation. Evaluated on motion planning, policy optimization, and system identification tasks, Dojo demonstrates significantly improved gradient accuracy and faster optimization convergence in challenging rigid-contact scenarios.

To address the challenge of enabling embodied agents to perform multi-step, multimodal complex tasks in dynamic physical environments, this paper introduces CLIER—the first closed-loop embodied reasoning system. CLIER employs a modular neuro-symbolic reasoning framework that jointly supports natural language instruction understanding, visual perception, non-visual property estimation (e.g., object weight), online belief updating, and action re-planning. We develop a MuJoCo-Blender co-simulation environment enabling high-fidelity physics interaction and photorealistic rendering. Furthermore, we establish the first benchmark encompassing ten categories of multi-step embodied reasoning tasks. Experiments demonstrate that CLIER achieves over 76% task success rate in simulation and 64% on a real robotic arm, significantly improving robustness against environmental disturbances, sensor noise, and actuation uncertainty, as well as cross-task generalization capability.

This work proposes a lightweight, open-source framework to address the challenges in robot learning associated with complex simulation deployment, heavy dependencies, and inefficient GPU acceleration. The framework uniquely integrates Isaac Lab’s manager-style API with the GPU-accelerated MuJoCo Warp physics engine, enabling modular composition of observation, reward, and event logic. It features one-command installation, minimal dependencies, and direct access to native MuJoCo data structures, significantly lowering the barrier to experimental setup. Reference implementations for velocity tracking, motion imitation, and manipulation tasks are provided, demonstrating improved simulation efficiency and ease of use.

To address the lack of flexible, composable simulation platforms and standardized evaluation protocols in robot learning, this paper introduces robosuite v1.0—the first modular robot learning simulation framework built on MuJoCo. Its core innovation is a novel modular task construction paradigm that decouples task specification, physics simulation, and reinforcement learning training via standardized APIs, reproducible benchmark environments, and ROS-compatible design. The framework provides over 12 benchmark tasks spanning bimanual manipulation, dexterous hand control, and multi-object interaction, supporting native Python invocation and cross-platform deployment. Since its release, robosuite v1.0 has been adopted in over 100 research studies, significantly advancing experimental reproducibility, comparability, and standardization in robot learning.

Existing approaches to online robot learning and batch physical simulation lack efficient, stateful parallel environment execution. This work proposes BatchEnvPool, which—without modifying MuJoCo’s core—introduces, for the first time, persistent, state-preserving batched runtime primitives that remain compatible with MuJoCo’s original CPU semantics while extending its rollout capabilities. Implemented in C++ with pybind11, the method employs an internal thread pool to manage multiple copies of mjModel and associated mjData workers, enabling short-step rollouts, sparse resets, domain randomization, and batched sensor and Jacobian queries. The open-source package mujoco-uni, installable via pip, demonstrably achieves substantial gains in parallel simulation throughput while fully preserving MuJoCo’s solver behavior and physical fidelity.

This work addresses the lack of scalable and reproducible simulation benchmarks for whole-body mobile manipulation in humanoid robots. We present the first unified simulation platform that supports large-scale tasks, diverse scenes, and extensive object assets, integrating MuJoCo’s high-fidelity dynamics with IsaacSim’s ray-traced rendering. The platform incorporates automated trajectory generation and low-latency VR-based teleoperation for real-world data collection. It enables, for the first time, visually realistic and contact-rich simulation of humanoid mobile manipulation, facilitating consistent evaluation of multiple state-of-the-art control policies. Experiments across 60 tasks and 50 scenes demonstrate strong sim-to-real correlation, with policies trained solely in simulation achieving zero-shot transfer to physical robots.

Existing quadrotor learning environments struggle to simultaneously achieve high physical fidelity, multi-agent support, and the high throughput required for deep reinforcement learning. To address this gap, this work introduces an open-source, Gymnasium-compatible multi-drone simulation platform built on MuJoCo, supporting an arbitrary number of Crazyflie nano-quadrotors. It presents the first highly modular multi-agent drone simulation framework within MuJoCo, integrating rigid-body dynamics, aerodynamic effects, and multimodal sensory inputs. The platform offers configurable physics models, action interfaces, and observation spaces, while leveraging GPU acceleration to enhance parallelization efficiency. Experimental results demonstrate its superiority over gym-pybullet-drones in contact handling, rendering quality, and training throughput, successfully reproducing and extending existing control and learning benchmarks.

Existing 3D datasets commonly lack the physical attributes necessary for stable and realistic interactions, necessitating extensive manual effort to construct simulation-ready articulated objects. This work introduces, for the first time, the concept of “interaction readiness” and establishes a quantifiable evaluation framework that decomposes its essential components. Building upon this foundation, the authors propose a method that combines multimodal perception fusion with closed-loop simulator optimization to automatically generate articulated objects with high-quality physical properties from incomplete 3D assets. By jointly reasoning over geometric, visual, and semantic cues to infer physical parameters and iteratively refining them through simulation feedback, the approach significantly enhances dynamic stability, interaction plausibility, and downstream policy learning performance across diverse manipulation tasks, while uncovering critical failure modes overlooked by conventional evaluation metrics.

Existing benchmarks for evaluating general-purpose robotic manipulation policies often suffer from simplistic tasks, limited evaluation dimensions, and a disconnect between simulation and real-world performance, hindering comprehensive assessment of policy capabilities. To address these limitations, this work introduces RoboDojo—the first standardized evaluation framework that enables coordinated simulation-to-reality benchmarking. RoboDojo encompasses 42 simulated and 18 real-world tasks, systematically evaluating policies across multiple dimensions including generalization, memory, precision, long-horizon execution, and open-vocabulary understanding. Built upon Isaac Sim for heterogeneous parallel simulation, the framework integrates the RoboDojo-RealEval physical evaluation system, the XPolicyLab unified policy interface, and cloud-based automated reset technology. It currently supports 30 diverse policies and features a public leaderboard, offering the community a reproducible and extensible infrastructure for robotic policy evaluation.