VoLUT: Efficient Volumetric streaming enhanced by LUT-based super-resolution

2025-02-17

AI Summary

To address the dual challenges of high bandwidth demand in 3D volumetric video streaming and the lack of volumetric-data-aware super-resolution (SR) algorithms, this paper proposes a lookup-table (LUT)-driven lightweight SR method specifically designed for volumetric video. The method enables low-overhead, high-fidelity reconstruction at the client side and integrates an adaptive bitrate (ABR) mechanism that dynamically adjusts the volumetric data downsampling ratio at the server to jointly optimize bandwidth efficiency and spatiotemporal consistency. To our knowledge, this is the first SR solution achieving lossless-quality, real-time 3D volumetric video upscaling on off-the-shelf mobile devices—filling a critical gap in volumetric-video-specific SR research. Experimental results demonstrate a 70% end-to-end bandwidth reduction and a 36.7% improvement in user quality-of-experience (QoE), significantly outperforming state-of-the-art general-purpose video SR approaches.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Time-Series/Data Streams
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

3D volumetric video provides immersive experience and is gaining traction in digital media. Despite its rising popularity, the streaming of volumetric video content poses significant challenges due to the high data bandwidth requirement. A natural approach to mitigate the bandwidth issue is to reduce the volumetric video's data rate by downsampling the content prior to transmission. The video can then be upsampled at the receiver's end using a super-resolution (SR) algorithm to reconstruct the high-resolution details. While super-resolution techniques have been extensively explored and advanced for 2D video content, there is limited work on SR algorithms tailored for volumetric videos. To address this gap and the growing need for efficient volumetric video streaming, we have developed VoLUT with a new SR algorithm specifically designed for volumetric content. Our algorithm uniquely harnesses the power of lookup tables (LUTs) to facilitate the efficient and accurate upscaling of low-resolution volumetric data. The use of LUTs enables our algorithm to quickly reference precomputed high-resolution values, thereby significantly reducing the computational complexity and time required for upscaling. We further apply adaptive video bit rate algorithm (ABR) to dynamically determine the downsampling rate according to the network condition and stream the selected video rate to the receiver. Compared to related work, VoLUT is the first to enable high-quality 3D SR on commodity mobile devices at line-rate. Our evaluation shows VoLUT can reduce bandwidth usage by 70% , boost QoE by 36.7% for volumetric video streaming and achieve 3D SR speed-up with no quality compromise.

Problem

• Reduces volumetric video streaming bandwidth requirements.
• Develops LUT-based super-resolution for 3D volumetric content.
• Enables high-quality 3D SR on mobile devices efficiently.

Innovation

• LUT-based super-resolution
• adaptive bit rate algorithm
• efficient volumetric streaming