A Multi-Agent DRL-Based Framework for Optimal Resource Allocation and Twin Migration in the Multi-Tier Vehicular Metaverse

2025-02-26

AI Summary

To address dynamic resource allocation and real-time vehicle twin (VT) migration in multi-layer vehicular metaverses, this paper proposes a synergistic optimization framework integrating graph convolutional networks (GCNs), hierarchical Stackelberg gaming, and multi-agent deep reinforcement learning (MADRL). We introduce MO-MADDPG—a novel algorithm that unifies GCN-based spatiotemporal dependency modeling, Stackelberg-driven vehicle-infrastructure coordination, and MADRL-enabled joint optimization of resource scheduling and VT migration. Formulated as a Markov decision process (MDP), the framework enables real-time, multi-objective trade-offs among latency, resource utilization, migration cost, and user experience under highly dynamic vehicular conditions. Experimental results demonstrate 12.8% latency reduction, 9.7% improvement in resource utilization, 14.2% lower migration cost, and 16.1% enhancement in user experience—collectively boosting system scalability, reliability, and operational efficiency.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Multiagent Systems: Adversarial Agents
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Although multi-tier vehicular Metaverse promises to transform vehicles into essential nodes -- within an interconnected digital ecosystem -- using efficient resource allocation and seamless vehicular twin (VT) migration, this can hardly be achieved by the existing techniques operating in a highly dynamic vehicular environment, since they can hardly balance multi-objective optimization problems such as latency reduction, resource utilization, and user experience (UX). To address these challenges, we introduce a novel multi-tier resource allocation and VT migration framework that integrates Graph Convolutional Networks (GCNs), a hierarchical Stackelberg game-based incentive mechanism, and Multi-Agent Deep Reinforcement Learning (MADRL). The GCN-based model captures both spatial and temporal dependencies within the vehicular network; the Stackelberg game-based incentive mechanism fosters cooperation between vehicles and infrastructure; and the MADRL algorithm jointly optimizes resource allocation and VT migration in real time. By modeling this dynamic and multi-tier vehicular Metaverse as a Markov Decision Process (MDP), we develop a MADRL-based algorithm dubbed the Multi-Objective Multi-Agent Deep Deterministic Policy Gradient (MO-MADDPG), which can effectively balances the various conflicting objectives. Extensive simulations validate the effectiveness of this algorithm that is demonstrated to enhance scalability, reliability, and efficiency while considerably improving latency, resource utilization, migration cost, and overall UX by 12.8%, 9.7%, 14.2%, and 16.1%, respectively.

Problem

• Optimize resource allocation in vehicular Metaverse.
• Enhance vehicular twin migration efficiency.
• Balance latency, resource use, and user experience.

Innovation

• GCNs capture spatial-temporal dependencies
• Stackelberg game fosters vehicular cooperation
• MADRL optimizes resource and VT migration