PhysPart: Physically Plausible Part Completion for Interactable Objects

2024-08-25

arXiv.org

AI Summary

This work addresses the problem of physically plausible completion of missing parts in interactive objects. We propose a diffusion-based generative method that, for the first time, explicitly incorporates physical constraints—such as structural stability and kinematic mobility—as differentiable loss terms within the diffusion sampling process. Our approach integrates classifier-free geometric guidance with a motion success rate evaluation mechanism, enabling part dependency modeling and hierarchical sequential generation. Quantitative evaluation demonstrates that our method achieves state-of-the-art performance across both geometric fidelity (e.g., Chamfer distance, F-Score) and physical plausibility metrics. Crucially, the newly introduced motion success rate metric empirically validates the high physical credibility of generated parts. Experiments confirm end-to-end applicability to downstream tasks including 3D printing, robotic manipulation, and modeling of complex assemblies.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Manipulation
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Interactable objects are ubiquitous in our daily lives. Recent advances in 3D generative models make it possible to automate the modeling of these objects, benefiting a range of applications from 3D printing to the creation of robot simulation environments. However, while significant progress has been made in modeling 3D shapes and appearances, modeling object physics, particularly for interactable objects, remains challenging due to the physical constraints imposed by inter-part motions. In this paper, we tackle the problem of physically plausible part completion for interactable objects, aiming to generate 3D parts that not only fit precisely into the object but also allow smooth part motions. To this end, we propose a diffusion-based part generation model that utilizes geometric conditioning through classifier-free guidance and formulates physical constraints as a set of stability and mobility losses to guide the sampling process. Additionally, we demonstrate the generation of dependent parts, paving the way toward sequential part generation for objects with complex part-whole hierarchies. Experimentally, we introduce a new metric for measuring physical plausibility based on motion success rates. Our model outperforms existing baselines over shape and physical metrics, especially those that do not adequately model physical constraints. We also demonstrate our applications in 3D printing, robot manipulation, and sequential part generation, showing our strength in realistic tasks with the demand for high physical plausibility.

Problem

• Physically plausible part completion
• Interactable object modeling
• Diffusion-based part generation

Innovation

• Diffusion-based part generation model
• Geometric conditioning with classifier-free guidance
• Physical constraints as stability and mobility losses