🤖 AI Summary
Existing 3D datasets commonly lack the physical attributes necessary for stable and realistic interactions, necessitating extensive manual effort to construct simulation-ready articulated objects. This work introduces, for the first time, the concept of “interaction readiness” and establishes a quantifiable evaluation framework that decomposes its essential components. Building upon this foundation, the authors propose a method that combines multimodal perception fusion with closed-loop simulator optimization to automatically generate articulated objects with high-quality physical properties from incomplete 3D assets. By jointly reasoning over geometric, visual, and semantic cues to infer physical parameters and iteratively refining them through simulation feedback, the approach significantly enhances dynamic stability, interaction plausibility, and downstream policy learning performance across diverse manipulation tasks, while uncovering critical failure modes overlooked by conventional evaluation metrics.
📝 Abstract
Simulation is a central tool for scalable robot learning, but its effectiveness depends on the quality of object assets. While modern 3D datasets provide rich geometric and kinematic representations, they typically lack the physical properties required for stable and realistic interaction, requiring significant manual effort to construct simulation-ready articulated objects. In this thesis, we introduce interaction-readiness, which characterizes whether an object can be reliably simulated under manipulation. We propose a quantitative evaluation framework that decomposes interaction-readiness into measurable components, enabling systematic analysis of object quality and revealing failure modes not captured by conventional evaluation. We further present a multi-modal, simulator-in-the-loop approach for generating interaction-ready articulated objects from incomplete 3D assets. The method integrates geometric, visual, and semantic information to infer physical properties and refines them through iterative simulator feedback to improve physical consistency. Experiments across diverse articulated objects and manipulation tasks show that object quality directly impacts simulation stability, interaction behavior, and policy performance. Objects refined by our method exhibit more stable and realistic dynamics, enabling more reliable downstream learning and evaluation. Overall, this thesis demonstrates the importance of physical realism for articulated objects in simulation and introduces a practical multi-modal refinement approach, guided by simulator feedback, for constructing such objects at scale.