PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects

2026-05-20

AI Summary

Existing 3D generation methods often neglect physical properties or are confined to a single object category, failing to meet the demand for diverse and physically plausible assets in downstream simulation tasks. This work proposes PhysX-Omni, a unified framework that achieves joint generation of rigid, deformable, and articulated 3D objects with physical fidelity for the first time. Key innovations include a compression-free, high-resolution geometric representation tailored for vision-language models, the first universal simulation-ready 3D dataset—PhysXVerse—and PhysX-Bench, a comprehensive evaluation benchmark encompassing six-dimensional physical attributes. Experiments demonstrate that the proposed method excels on both conventional metrics and PhysX-Bench, significantly enhancing performance in downstream applications such as simulation scene generation and robot policy learning.

Abstract

Simulation-ready physical 3D assets have emerged as a promising direction owing to their broad applicability in downstream tasks. However, most existing 3D generation methods either neglect physical properties or are limited to a single asset category, e.g., rigid, deformable, or articulated objects. To address these limitations, we introduce PhysX-Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types. Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression, significantly improving generation performance. In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories. Furthermore, to comprehensively and flexibly evaluate both generative and understanding capabilities in the wild, we propose PhysX-Bench, which encompasses six key attributes: geometry, absolute scale, material, affordance, kinematics, and function description. Extensive experiments with conventional metrics and PhysX-Bench show that PhysX-Omni performs strongly in both generation and understanding. Moreover, additional studies further validate the potential of PhysX-Omni for applications in simulation-ready scene generation and robotic policy learning. We believe PhysX-Omni can significantly advance a wide range of downstream applications, particularly in embodied AI and physics-based simulation.

Problem

• simulation-ready
• physical 3D generation
• rigid objects
• deformable objects
• articulated objects

Innovation

• simulation-ready 3D generation
• unified physical representation
• Vision-Language Models
• PhysXVerse dataset
• PhysX-Bench evaluation