Enhanced Neural Video Representation Compression across Extreme Complexity and Quality Scales

2026-06-26

AI Summary

Existing neural video codecs struggle to balance complexity and scalability: lightweight models suffer from limited rate-distortion performance, while high-performance models exhibit rapidly increasing computational complexity with quality, and lack a unified architecture for multi-scenario deployment. This work proposes NVRC++, a unified neural video coding framework based on implicit neural representations (INRs). By integrating a lightweight INR, multi-resolution feature grids, an efficient overfitting-aware optimization tailored for long videos, and an advanced high-dimensional entropy model, NVRC++ supports four complexity levels (7k–360k MACs/pixel) within a single fixed architecture. Each level spans a broad bitrate range while maintaining real-time decoding. Compared to the state-of-the-art NVRC, NVRC++ achieves up to 7.6× faster decoding at comparable rate-distortion performance, marking the first neural video compression system capable of efficient operation across extreme scales of both complexity and quality.

Abstract

Implicit neural representations (INRs) have recently emerged as a promising approach to video compression, delivering competitive rate-distortion performance alongside rapid decoding. However, existing neural video codecs struggle to balance complexity and scalability. Lightweight models often suffer from degraded compression performance when scaled to different bitrate/quality levels, whereas high-performance models exhibit limited scalability, as their model complexity typically increases with quality. This lack of a unified architecture capable of maintaining consistent complexity across a wide range of bitrates severely limits their diverse real-world deployment. To address these challenges, we introduce NVRC++, a novel INR-based video codec that utilizes a lightweight INR with multiple high-resolution feature grids, providing high scalability at any given complexity level. This is paired with an optimization framework that enables efficient overfitting on high-resolution grids for long video sequences, thereby exploiting spatio-temporal redundancies without prohibitive computational or memory overhead. Additionally, an advanced entropy model is designed for efficiently compressing the high-dimensional grid parameters. As a result, NVRC++ provides four complexity levels (from 7kMACs/pixel to 360kMACs/pixel), each spanning wide bitrate and quality ranges while supporting real-time decoding. The experimental results show that NVRC++ offers a much faster decoding speed (up to 7.6x) compared to the SOTA INR-based video codec, NVRC, while delivering comparable performance.

Problem

• neural video compression
• implicit neural representations
• complexity scalability
• rate-distortion performance
• video codec

Innovation

• Implicit Neural Representations
• Video Compression
• Scalable Codec
• Feature Grids
• Entropy Modeling