URDF Synthesis from RGB-D Sequences via Differentiable Joint Inference and Energy-Consistent Verification

2026-06-17

AI Summary

This work addresses the challenge of reconstructing simulation-ready digital twins of articulated objects from RGB-D sequences, where existing methods often suffer from simulation drift due to decoupled geometry and kinematics estimation and the neglect of dynamic constraints. The authors propose KinemaForge, a novel framework that jointly optimizes part geometry, joint topology, and parameters, and introduces an energy-consistency verification mechanism grounded in differentiable rigid-body dynamics. By integrating a kinematic constraint graph, a differentiable screw-axis solver, and the Featherstone algorithm, the approach enforces physical plausibility through an energy conservation loss that enables closed-loop validation. Experiments demonstrate significant improvements: joint axis errors are reduced by 37.4%–46.6%, simulation drift over 50 seconds decreases by 64%, and closed-loop manipulation success rate increases by 14.6 percentage points.

Abstract

Reconstructing simulation-ready digital twins of articulated objects from sensor observations remains constrained by two persistent gaps: (i) part-level geometric reconstruction is decoupled from kinematic-parameter estimation, and (ii) the recovered models often violate basic dynamic invariants such as energy conservation, leading to drift when the URDF is replayed in physics simulators. We present KinemaForge, a constraint-driven pipeline that jointly infers part-level shape, joint topology, and joint parameters from short RGB-D sequences and validates the result against an energy-consistent verifier built on differentiable rigid-body dynamics. The pipeline introduces three components: a kinematic constraint graph that encodes joint-part incidences as soft edges; a differentiable screw-axis solver that backpropagates from rendered observations through Featherstone's articulated-body algorithm to joint parameters; and an energy residual loss that penalises non-physical free responses of the reconstructed model. Across five PartNet-Mobility categories and an internal RGB-D benchmark, KinemaForge reduces the average joint-axis error from 4.52 degrees to 2.83 degrees (-37.4%) over the strongest geometric baseline (PARIS) and from 5.30 degrees to 2.83 degrees (-46.6%) over the interaction-based Ditto baseline, lowers long-horizon simulation drift by 64% (vs. PARIS) over 50 s rollouts, and yields URDFs whose closed-loop manipulation success rate improves by 14.6 percentage points over Ditto in our preliminary evaluation. Code and reconstruction data will be released upon acceptance.

Problem

• articulated objects
• URDF synthesis
• energy conservation
• kinematic-parameter estimation
• simulation drift

Innovation

• differentiable dynamics
• energy-consistent verification
• joint parameter inference
• URDF synthesis
• articulated object reconstruction