Semantic Video Communication via Multi-Scale Convolution and Dynamic Routing for Next-Generation Networks

2026-07-06

AI Summary

This work addresses the challenge of efficiently transmitting video semantic information under bandwidth and edge resource constraints by proposing a generative AI–driven semantic video communication framework. The approach employs a multi-scale temporal convolutional encoder with O(T) complexity to model motion patterns at multiple granularities and introduces, for the first time, a capsule network–based dynamic routing mechanism to enable non-monotonic and robust cross-modal alignment between video segments and natural language queries. The framework is unified under a multi-task learning objective that jointly optimizes temporal boundary regression, semantic alignment, and capsule diversity. Evaluated on ActivityNet Captions, it achieves 42.9% Recall@0.5 and 41.1% mean IoU, demonstrating a strong balance between semantic accuracy and edge deployment efficiency.

Abstract

The exponential growth of video traffic demands novel semantic communication paradigms that transmit meaning rather than raw bits. We present a generative AI-enabled framework for semantic video communication addressing two critical challenges: efficient hierarchical temporal modeling for bandwidth-constrained transmission and robust semantic alignment between video content and natural language queries at network edge devices. Our approach introduces a multi-scale temporal convolutional encoder that captures motion patterns across different temporal granularities with O(T) complexity suitable for resource-constrained IoT deployments. We further propose a capsule-based dynamic routing mechanism that iteratively refines segment-query associations, enabling flexible modeling of non-monotonic semantic alignments essential for goal-oriented communication. These components are unified through a multi-task learning objective optimizing temporal boundary regression, cross-modal alignment, and capsule diversity. Experiments on ActivityNet Captions demonstrate significant improvements, achieving 42.9% Recall@0.5 and 41.1% mean IoU while maintaining computational efficiency critical for edge deployment.

Problem

• semantic video communication
• hierarchical temporal modeling
• semantic alignment
• bandwidth-constrained transmission
• edge devices

Innovation

• semantic video communication
• multi-scale temporal convolution
• dynamic routing
• capsule network
• edge AI