DeformSmith: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation

2026-09-16

AI Summary

为解决机器人操作中变形物体的创建难题，DeformSmith通过基于物理的引导和层次化生成方法，从文本或单张图片自动生成具有交互性和物理可信度的变形物体。

Abstract

Creating deformable assets for robot manipulation requires jointly specifying their geometry, appearance, and physical properties. This is especially challenging for deformable objects, since text and images provide limited evidence about how they deform and respond to contact, yet these responses directly affect their suitability for interaction. Automated generation therefore needs to resolve coupled physical requirements and use interaction evidence to guide construction and refinement. We present DeformSmith, a framework that enables automated generation of interactive, physically credible deformable assets from text or a single image. Through hierarchical agentic construction and a shared physics-grounded harness, it progressively builds, tests, and refines geometry, physical models, material behavior, and robot interaction until the resulting asset is ready for simulation and manipulation. Robot interaction closes the generation loop through manipulation feedback and replayable interaction data. Results show that DeformSmith generates assets with better visual quality and physical plausibility than state-of-the-art baselines, including PhysGen3D, PhysGM, and PhysX-Omni, while supporting the synthesis of data for robotic manipulation of deformable objects. Project page: https://can-lee.github.io/deformsmith-web/

Problem

• deformable objects
• robot manipulation
• physical properties
• geometry
• appearance

Innovation

• hierarchical agentic construction
• physics-grounded harness
• robot manipulation feedback