Foveated Compression for Immersive Telepresence Visualization

2025-10-21

AI Summary

To address bandwidth limitations in immersive remote visualization—causing degraded resolution and fidelity—this paper proposes an eye-tracking-driven foveated compression method. It integrates real-time eye-tracking data with standard video encoders (e.g., H.264/AVC) to dynamically modulate spatial-domain quantization parameters: preserving high fidelity in the foveal region while substantially reducing bitrates in peripheral areas, thereby achieving retinal-perception-aligned non-uniform compression. We design a lightweight, plug-and-play foveated compression framework and present its first end-to-end validation on the award-winning NimbRo Avatar system. Experiments demonstrate that, without compromising subjective immersion, average bandwidth consumption drops to 33% of the original stream. The approach thus achieves an optimal trade-off between compression efficiency and visual quality, establishing a novel paradigm for bandwidth-constrained telepresence and teleoperation applications.

Technology Category

• Computer Vision: Motion & Tracking
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Data Mining & Knowledge Management: Data Compression

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Immersive televisualization is important both for telepresence and teleoperation, but resolution and fidelity are often limited by communication bandwidth constraints. We propose a lightweight method for foveated compression of immersive televisualization video streams that can be easily integrated with common video codecs, reducing the required bandwidth if eye tracking data is available. Specifically, we show how to spatially adjust the Quantization Parameter of modern block-based video codecs in a adaptive way based on eye tracking information. The foveal region is transmitted with high fidelity while quality is reduced in the peripheral region, saving bandwidth. We integrate our method with the NimbRo avatar system, which won the ANA Avatar XPRIZE competition. Our experiments show that bandwidth can be reduced to a third without sacrificing immersion. We analyze transmission fidelity with qualitative examples and report quantitative results.

Problem

• Reducing bandwidth for immersive telepresence video streams
• Adapting video compression based on eye tracking data
• Maintaining high foveal quality while reducing peripheral fidelity

Innovation

• Foveated compression adjusts video quality spatially
• Uses eye tracking to guide quantization parameter adaptation
• Integrates with block-based codecs to reduce bandwidth