Streaming of rendered content with adaptive frame rate and resolution

2026-05-09

AI Summary

This work addresses the limitations of existing bandwidth-constrained streaming rendering approaches, which adapt only resolution while fixing frame rate, resulting in suboptimal perceptual quality and high rendering overhead. The paper presents the first joint adaptive control mechanism that simultaneously modulates both frame rate and resolution. Leveraging the spatiotemporal characteristics of the human visual system and incorporating scene content and motion dynamics, a lightweight neural network predicts the optimal parameter configuration in real time. The proposed method is codec-agnostic and fully compatible with existing rendering pipelines. Experimental results demonstrate that, under identical bandwidth constraints, it achieves significantly improved subjective visual quality while substantially reducing computational load on the server side.

Abstract

Streaming rendered content is an attractive way to bring high-quality graphics to billions of mobile devices that do not have sufficient rendering power. Existing solutions render content on a server at a fixed frame rate, typically 30 or 60 frames per second, and reduce resolution when bandwidth is restricted. However, this strategy leads to suboptimal rendering quality under the bandwidth constraints. In this work, we exploit the spatio-temporal limits of the human visual system to improve perceived quality while reducing rendering costs by adaptively adjusting both frame rate and resolution based on scene content and motion. Our approach is codec-agnostic and requires only minimal modifications to existing rendering infrastructure. We propose a system in which a lightweight neural network predicts the optimal combination of frame rate and resolution for a given transmission bandwidth, content, and motion velocity. This prediction significantly enhances perceptual quality while minimizing computational cost under bandwidth constraints. The network is trained on a large dataset of rendered content labeled with a perceptual video quality metric. The dataset and further information can be found at the project web page: https://www.cl.cam.ac.uk/research/rainbow/projects/adaptive_streaming/.

Problem

• adaptive streaming
• frame rate adaptation
• resolution adaptation
• perceptual quality
• rendered content

Innovation

• adaptive streaming
• frame rate adaptation
• resolution scaling
• perceptual quality optimization
• neural network prediction