Demonstrating DVS: Dynamic Virtual-Real Simulation Platform for Mobile Robotic Tasks

2025-04-26

AI Summary

Existing robotic simulation platforms suffer from overly idealized environments, limited task diversity, poor data interoperability, and inadequate capabilities in dynamic pedestrian modeling, editable scene construction, and virtual–physical asset synchronization—hindering complex task training and real-world deployment. This paper introduces DVS (Dynamic Virtual–Physical Synchronization), a novel simulation platform. DVS pioneers an optical motion-capture–driven real-time pose and coordinate mapping mechanism between virtual and physical domains. It supports editable large-scale indoor scenes, stochastic pedestrian behavior modeling, and closed-loop validation with human-in-the-loop intervention. Furthermore, DVS establishes a unified multi-task benchmark covering trajectory prediction, path planning, and robotic arm grasping. Experiments demonstrate that DVS significantly enhances cross-domain generalization and enables seamless transfer of simulation-trained models to real robots.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

With the development of embodied artificial intelligence, robotic research has increasingly focused on complex tasks. Existing simulation platforms, however, are often limited to idealized environments, simple task scenarios and lack data interoperability. This restricts task decomposition and multi-task learning. Additionally, current simulation platforms face challenges in dynamic pedestrian modeling, scene editability, and synchronization between virtual and real assets. These limitations hinder real world robot deployment and feedback. To address these challenges, we propose DVS (Dynamic Virtual-Real Simulation Platform), a platform for dynamic virtual-real synchronization in mobile robotic tasks. DVS integrates a random pedestrian behavior modeling plugin and large-scale, customizable indoor scenes for generating annotated training datasets. It features an optical motion capture system, synchronizing object poses and coordinates between virtual and real world to support dynamic task benchmarking. Experimental validation shows that DVS supports tasks such as pedestrian trajectory prediction, robot path planning, and robotic arm grasping, with potential for both simulation and real world deployment. In this way, DVS represents more than just a versatile robotic platform; it paves the way for research in human intervention in robot execution tasks and real-time feedback algorithms in virtual-real fusion environments. More information about the simulation platform is available on https://immvlab.github.io/DVS/.

Problem

• Limited simulation platforms for complex robotic tasks
• Challenges in dynamic pedestrian and scene modeling
• Lack of virtual-real synchronization for robot deployment

Innovation

• Dynamic virtual-real synchronization platform for robotics
• Random pedestrian behavior modeling plugin integration
• Optical motion capture for virtual-real pose synchronization