Semantic-Aware Neural Video Codec for Error-Resilient Low-Latency Transmission

2026-09-14

AI Summary

该研究提出一种语义感知的多级神经视频编码方法，通过将编码表示分配到不同优先级的数据流中，并采用抗错误熵模型，以实现在不可靠信道上的鲁棒低延迟视频传输。

Abstract

Emerging physical AI systems require low-latency, task-oriented video communication over unreliable channels. We propose a semantic-aware multi-level neural video coding method for robust low-latency video transmission over unreliable channels that are abstracted as multi-level packet erasure channels. Built upon the real-time DCVC-RT neural video codec, the proposed framework introduces a semantic- and feature-aware coding strategy that partitions encoded representations into packets carrying different levels of semantic and latent-feature importance and assigns these packets to different streams, each associated with a priority level when transmitted over unreliable communication channels. We also developed an error-resilient entropy model that removes inter-packet dependencies, allowing each packet to be decoded independently under packet losses. The complete system is trained end-to-end over the abstracted multi-level packet erasure channels, enabling learning of channel-aware representations together with importance-aware packet assignment while facilitating the network for differentiated packet prioritization. Experiments show that the proposed framework significantly improves robustness over baseline DCVC-RT under packet erasures, achieving graceful degradation in less important regions while better preserving task-relevant visual content.

Problem

• low-latency
• unreliable channels
• video communication
• error-resilient
• semantic-aware

Innovation

• semantic-aware coding
• multi-level neural video coding
• error-resilient entropy model
• packet prioritization