WVSC: Wireless Video Semantic Communication with Multi-frame Compensation

2025-03-27

AI Summary

Conventional pixel-level video coding for wireless transmission neglects semantic redundancy, resulting in suboptimal bandwidth efficiency. Method: This paper proposes the first end-to-end semantic-level video communication framework, featuring: (1) a deep semantic encoder that maps video frames to compact semantic representations; (2) reference semantic frames—replacing traditional motion vectors—to eliminate pixel-level alignment dependencies; and (3) a multi-frame compensation (MFC) attention module enabling cross-frame fusion and reconstruction directly in the semantic domain. Contribution/Results: It is the first work to systematically integrate the semantic communication paradigm into wireless video transmission, achieving joint source-channel optimization. Experiments demonstrate that the proposed method achieves approximately 1 dB higher PSNR than DVSC and about 2 dB higher than H.264/HEVC, significantly improving both reconstruction quality and bandwidth efficiency.

Technology Category

• Computer Vision: Multi-modal Vision
• Multiagent Systems: Agent Communication
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Existing wireless video transmission schemes directly conduct video coding in pixel level, while neglecting the inner semantics contained in videos. In this paper, we propose a wireless video semantic communication framework, abbreviated as WVSC, which integrates the idea of semantic communication into wireless video transmission scenarios. WVSC first encodes original video frames as semantic frames and then conducts video coding based on such compact representations, enabling the video coding in semantic level rather than pixel level. Moreover, to further reduce the communication overhead, a reference semantic frame is introduced to substitute motion vectors of each frame in common video coding methods. At the receiver, multi-frame compensation (MFC) is proposed to produce compensated current semantic frame with a multi-frame fusion attention module. With both the reference frame transmission and MFC, the bandwidth efficiency improves with satisfying video transmission performance. Experimental results verify the performance gain of WVSC over other DL-based methods e.g. DVSC about 1 dB and traditional schemes about 2 dB in terms of PSNR.

Problem

• Enables semantic-level video coding for wireless transmission
• Reduces communication overhead with reference semantic frames
• Improves bandwidth efficiency via multi-frame compensation

Innovation

• Semantic-level video coding replaces pixel-level coding
• Reference semantic frame reduces communication overhead
• Multi-frame compensation enhances bandwidth efficiency