SyncTwin: Fast Digital Twin Construction and Synchronization for Safe Robotic Grasping

2026-01-14

AI Summary

This work proposes a closed-loop digital twin framework to address the challenges of accuracy and safety in robotic grasping under dynamically occluded visual conditions. The framework integrates an offline 3D asset library with online state synchronization, leveraging VGGT to rapidly reconstruct 3D models from RGB images. By combining point cloud segmentation with color-based ICP registration, it achieves real-time alignment between the physical and simulated environments. This synchronized digital twin enables robust grasp planning and safe execution. Experimental results demonstrate that the proposed method significantly improves both grasping success rates and motion safety in dynamic occlusion scenarios, validating the effectiveness and novelty of the digital twin mechanism for real-world robotic tasks.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance

Abstract

Accurate and safe grasping under dynamic and visually occluded conditions remains a core challenge in real-world robotic manipulation. We present SyncTwin, a digital twin framework that unifies fast 3D scene reconstruction and real-to-sim synchronization for robust and safety-aware grasping in such environments. In the offline stage, we employ VGGT to rapidly reconstruct object-level 3D assets from RGB images, forming a reusable geometry library for simulation. During execution, SyncTwin continuously synchronizes the digital twin by tracking real-world object states via point cloud segmentation updates and aligning them through colored-ICP registration. The updated twin enables motion planners to compute collision-free and dynamically feasible trajectories in simulation, which are safely executed on the real robot through a closed real-to-sim-to-real loop. Experiments in dynamic and occluded scenes show that SyncTwin improves grasp accuracy and motion safety, demonstrating the effectiveness of digital-twin synchronization for real-world robotic execution.

Problem

• robotic grasping
• dynamic environments
• visual occlusion
• safety
• digital twin

Innovation

• Digital Twin
• 3D Reconstruction
• Real-to-Sim Synchronization
• Robotic Grasping
• Colored-ICP