Causal-Aware Intelligent QoE Optimization for VR Interaction with Adaptive Keyframe Extraction

2025-06-24

AI Summary

To address the challenge of jointly achieving low latency, high-fidelity motion synchronization, and fair resource allocation in multi-user VR interactions, this paper proposes a causality-aware joint resource optimization framework. Methodologically, it integrates adaptive keyframe extraction, Weber–Fechner law-based perceptual sensitivity modeling for Quality of Experience (QoE), and a causality-guided Deep Deterministic Policy Gradient (DDPG) algorithm; further, it incorporates causal influence detection and an attention-driven priority mechanism, enabling coordinated scheduling of bandwidth, CPU frequency, and keyframe policies via mixed-integer programming. Evaluated on the CMU Motion Capture dataset, the framework reduces interaction latency by 23.6% and improves QoE by 19.4% over baseline methods, while significantly enhancing inter-user perceptual fairness. The core contribution lies in embedding causal reasoning into RL action exploration and establishing a perception-driven dynamic resource allocation paradigm.

Technology Category

• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures

Abstract

The optimization of quality of experience (QoE) in multi-user virtual reality (VR) interactions demands a delicate balance between ultra-low latency, high-fidelity motion synchronization, and equitable resource allocation. While adaptive keyframe extraction mitigates transmission overhead, existing approaches often overlook the causal relationships among allocated bandwidth, CPU frequency, and user perception, limiting QoE gains. This paper proposes an intelligent framework to maximize QoE by integrating adaptive keyframe extraction with causal-aware reinforcement learning (RL). First, a novel QoE metric is formulated using the Weber-Fechner Law, combining perceptual sensitivity, attention-driven priorities, and motion reconstruction accuracy. The QoE optimization problem is then modeled as a mixed integer programming (MIP) task, jointly optimizing keyframe ratios, bandwidth, and computational resources under horizon-fairness constraints. We propose Partial State Causal Deep Deterministic Policy Gradient (PS-CDDPG), which integrates the Deep Deterministic Policy Gradient (DDPG) method with causal influence detection. By leveraging causal information regarding how QoE is influenced and determined by various actions, we explore actions guided by weights calculated from causal inference (CI), which in turn improves training efficiency. Experiments conducted with the CMU Motion Capture Database demonstrate that our framework significantly reduces interactive latency, enhances QoE, and maintains fairness, achieving superior performance compared to benchmark methods.

Problem

• Optimize QoE in VR with adaptive keyframe extraction
• Address causal links in bandwidth, CPU, and perception
• Enhance QoE via causal-aware reinforcement learning

Innovation

• Causal-aware reinforcement learning for QoE optimization
• Adaptive keyframe extraction with Weber-Fechner Law
• Partial State Causal DDPG for efficient training