Radiance Field Delta Video Compression in Edge-Enabled Vehicular Metaverse

2024-11-01

arXiv.org

AI Summary

To address bandwidth constraints and insufficient robustness against dynamic environmental conditions in physical-to-virtual (P2V) real-time synchronization for vehicle-infrastructure cooperative metaverses, this paper proposes a radiance field (RF)-based incremental video compression method. It encodes only dynamic traffic changes by comparing onboard captured frames with RF-rendered frames from a digital twin empty scene, enabling low-latency, high-fidelity P2V synchronization. We introduce the first integrated paradigm that tightly couples RF modeling with video compression for incremental encoding, and the first RF-based P2V compression framework supporting robust operation under complex lighting, rain, and fog conditions in vehicular edge networks. Leveraging distributed digital twin modeling, incremental frame-difference extraction, MEC-coordinated transmission, and URLLC-adaptive optimization, our method reduces bandwidth consumption by 71% and 44% compared to H.264 and H.265, respectively. Under BLER = 0.35 (clear) and 0.2 (rain), SSIM improves by 0.29 and 0.25, significantly enhancing error resilience and visual fidelity.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Connected and autonomous vehicles (CAVs) offload computationally intensive tasks to multi-access edge computing (MEC) servers via vehicle-to-infrastructure (V2I) communication, enabling applications within the vehicular metaverse, which transforms physical environment into the digital space enabling advanced analysis or predictive modeling. A core challenge is physical-to-virtual (P2V) synchronization through digital twins (DTs), reliant on MEC networks and ultra-reliable low-latency communication (URLLC). To address this, we introduce radiance field (RF) delta video compression (RFDVC), which uses RF-encoder and RF-decoder architecture using distributed RFs as DTs storing photorealistic 3D urban scenes in compressed form. This method extracts differences between CAV-frame capturing actual traffic and RF-frame capturing empty scene from the same camera pose in batches encoded and transmitted over the MEC network. Experiments show data savings up to 71% against H.264 codec and 44% against H.265 codec under different conditions as lighting changes, and rain. RFDVC also demonstrates resilience to transmission errors, achieving up to +0.29 structural similarity index measure (SSIM) improvement at block error rate (BLER) = 0.35 in non-rainy and +0.25 at BLER = 0.2 in rainy conditions, ensuring superior visual quality compared to standard video coding (VC) methods across various conditions.

Problem

• Synchronization Challenge
• Data Transmission Efficiency
• Complex Computational Tasks

Innovation

• Radiation Field (RF) DVC
• Data Efficiency
• Error Resilience