SOPHY: Generating Simulation-Ready Objects with Physical Materials

2025-04-17

AI Summary

Existing 3D generative models struggle to jointly model geometry, appearance, and physically grounded material properties—hindering direct use in physics simulation and dynamic interaction. This paper introduces SOPHY, the first end-to-end, physically simulatable 3D generation framework supporting both text-driven synthesis and single-image reconstruction. Its key contributions are: (1) the first unified modeling of shape, texture, and dynamics-relevant material properties—including Young’s modulus, density, and coefficient of friction; (2) the construction of the first 3D dataset with fine-grained physical material annotations, enabled by an automated annotation pipeline; and (3) the integration of physics-constrained material representation learning, multimodal conditional modeling, and physics-aware geometric optimization. Experiments demonstrate that SOPHY significantly outperforms prior methods in geometric plausibility and visual fidelity, while enabling out-of-the-box rigid-body and soft-body simulation and interactive physics-based manipulation.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

We present SOPHY, a generative model for 3D physics-aware shape synthesis. Unlike existing 3D generative models that focus solely on static geometry or 4D models that produce physics-agnostic animations, our approach jointly synthesizes shape, texture, and material properties related to physics-grounded dynamics, making the generated objects ready for simulations and interactive, dynamic environments. To train our model, we introduce a dataset of 3D objects annotated with detailed physical material attributes, along with an annotation pipeline for efficient material annotation. Our method enables applications such as text-driven generation of interactive, physics-aware 3D objects and single-image reconstruction of physically plausible shapes. Furthermore, our experiments demonstrate that jointly modeling shape and material properties enhances the realism and fidelity of generated shapes, improving performance on generative geometry evaluation metrics.

Problem

• Generates 3D objects with physics-ready materials
• Synthesizes shape, texture, and dynamic material properties
• Enables text-driven and image-based physics-aware 3D generation

Innovation

• Generates 3D objects with physics materials
• Jointly synthesizes shape, texture, material
• Uses dataset with physical attributes