Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decomposition

2026-08-24

AI Summary

为解决生成的3D网格缺乏物理属性的问题，Gen2Physics通过多视图材料分解方法将这些网格转换为具有物理特性的模拟就绪资产。

Abstract

While state-of-the-art generative models produce high-fidelity 3D meshes, these outputs lack the physical properties required for interactive simulation, gaming, or robotics. We introduce Gen2Physics, a unified and automated framework that grounds generated meshes in physics by automatically decomposing them into their constituent material components. Unlike prior approaches, which focus on volumetric representations incompatible with standard physics engines, Gen2Physics operates directly on meshes to produce immediately simulation-ready assets. Our pipeline integrates a fine-tuned Vision Transformer for dense material segmentation, a robust 2D-to-3D consistency projection, and a Vision-Language Model (VLM) guided refinement that leverages contextual reasoning to assign physical properties and infer internal geometry (solid vs. hollow). By converting surface patches into volumes with distinct densities, our method enables physically plausible dynamic simulations. Experimental results on the ABO-500 and PartNet-Material benchmarks demonstrate that Gen2Physics more than doubles the material segmentation accuracy of prior physics-grounding pipelines (15.6 to 48.3 mIoU), while matching the mass-estimation accuracy of volumetric methods and being the only approach to output watertight per-material sub-meshes.

Problem

• 3D meshes
• physical properties
• interactive simulation
• material decomposition
• physics engines

Innovation

• Vision Transformer
• Material Segmentation
• Simulation-Ready Assets
• Physical Properties Assignment
• Volume Conversion