ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos

2026-06-23

AI Summary

Deploying robots in unstructured real-world environments demands high-fidelity, interactive object models, yet existing approaches rely on CAD assets, simulation, or manual annotations, limiting scalability. This work proposes the first end-to-end framework that integrates 3D Gaussian splatting with unsupervised articulation discovery to automatically reconstruct interactive digital twins from RGB-D video alone. The resulting models capture accurate geometry, photometric detail, and articulated kinematics without requiring simulation or human annotation. The method enables real-time rendering, viewpoint control, and physical interaction, and can be directly integrated into robotic planning and learning systems. By eliminating the need for synthetic data or labeled supervision, this approach significantly lowers the barrier to modeling articulated objects for embodied intelligence tasks.

Abstract

Deploying robots in unstructured real-world environments needs accurate, interactive models of the objects. Constructing these models at scale remains a critical bottleneck for robotic system integration. We present ArtiTwinSplat, a framework that automatically constructs articulated, photo-realistic digital twins of objects directly from RGB-D videos, requiring no CAD models, simulation assets, or manual annotations. Our method is built on 3D Gaussian Splatting that preserve geometric fidelity and photometric realism, coupled with an unsupervised articulation discovery pipeline that recovers part structure and joint kinematics from observed motion alone. With tracking and optimization stages our method provides stable, queryable digital twins that support real-time rendering, viewpoint control, and interactive manipulation. Unlike prior methods confined to simulation, ArtiTwinSplat operates directly on real-world observations and produces twins that are immediately usable by downstream robot planning and learning systems. This method offers a practical, scalable pathway toward digital twin construction, lowering the integration barrier for articulated object manipulation in embodied AI and human-robot collaboration contexts.

Problem

• digital twin
• articulated objects
• robotic manipulation
• RGB-D videos
• embodied AI

Innovation

• Gaussian Splatting
• Digital Twin
• Articulated Objects
• Unsupervised Articulation Discovery
• RGB-D Video