Multi-Source Human-in-the-Loop Digital Twin Testbed for Connected and Autonomous Vehicles in Mixed Traffic Flow

2026-03-18

AI Summary

This work addresses the need for a highly flexible, multi-source human-in-the-loop testing platform to study complex interactions between connected and automated vehicles (CAVs) and human-driven vehicles (HDVs) in mixed traffic flows. To this end, the authors propose the MSH-MCCT testbed, which uniquely integrates mixed reality and digital twin technologies to create a cyber-physical environment where physical, virtual, and mixed-reality entities coexist. The platform supports the integration of multi-fidelity driving simulators and enables real-time, multi-perspective collaborative interaction. It achieves seamless coexistence of physical and virtual CAVs/HDVs, significantly enhancing testing flexibility, scalability, and human-in-the-loop diversity. The effectiveness of the platform is demonstrated through platooning experiments in mixed traffic, confirming its capability to involve multiple real human drivers in the evaluation of CAV algorithms.

Technology Category

• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Computer Vision: Multi-modal Vision

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

In the emerging mixed traffic environments, Connected and Autonomous Vehicles (CAVs) have to interact with surrounding human-driven vehicles (HDVs). This paper introduces MSH-MCCT (Multi-Source Human-in-the-Loop Mixed Cloud Control Testbed), a novel CAV testbed that captures complex interactions between various CAVs and HDVs. Utilizing the Mixed Digital Twin concept, which combines Mixed Reality with Digital Twin, MSH-MCCT integrates physical, virtual, and mixed platforms, along with multi-source control inputs. Bridged by the mixed platform, MSH-MCCT allows human drivers and CAV algorithms to operate both physical and virtual vehicles within multiple fields of view. Particularly, this testbed facilitates the coexistence and real-time interaction of physical and virtual CAVs \& HDVs, significantly enhancing the experimental flexibility and scalability. Experiments on vehicle platooning in mixed traffic showcase the potential of MSH-MCCT to conduct CAV testing with multi-source real human drivers in the loop through driving simulators of diverse fidelity. The videos for the experiments are available at our project website: https://dongjh20.github.io/MSH-MCCT.

Problem

• Connected and Autonomous Vehicles
• Mixed Traffic Flow
• Human-in-the-Loop
• Digital Twin
• Human-Driven Vehicles

Innovation

• Mixed Digital Twin
• Human-in-the-Loop
• Connected and Autonomous Vehicles
• Mixed Reality
• Multi-Source Testbed