Multi-Task Semantic Communication With Graph Attention-Based Feature Correlation Extraction

2025-01-02

IEEE Transactions on Mobile Computing

AI Summary

To address insufficient cross-task semantic feature correlation modeling in multi-task semantic communication under a shared encoder, this paper proposes the Graph Attention Interaction (GAI) module. GAI explicitly represents intermediate-layer semantic features from the encoder as graph nodes and employs graph attention mechanisms to dynamically learn inter-task feature dependencies, further refined via MLP-based fine-grained collaborative enhancement. Under an extremely stringent bandwidth compression ratio of 1/12, GAI significantly improves semantic representation capability. On the CityScapes 2Task and NYU V2 3Task benchmarks, it achieves absolute improvements of 11.4% and 3.97% over state-of-the-art methods, respectively. These results demonstrate GAI’s dual advantages in enhancing multi-task collaboration and improving resource efficiency—particularly critical for bandwidth-constrained semantic communication systems.

Technology Category

• Computer Vision: Multi-modal Vision
• Multiagent Systems: Agent Communication
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Multi-task semantic communication can serve multiple learning tasks using a shared encoder model. Existing models have overlooked the intricate relationships between features extracted during an encoding process of tasks. This paper presents a new graph attention inter-block (GAI) module to the encoder/transmitter of a multi-task semantic communication system, which enriches the features for multiple tasks by embedding the intermediate outputs of encoding in the features, compared to the existing techniques. The key idea is that we interpret the outputs of the intermediate feature extraction blocks of the encoder as the nodes of a graph to capture the correlations of the intermediate features. Another important aspect is that we refine the node representation using a graph attention mechanism to extract the correlations and a multi-layer perceptron network to associate the node representations with different tasks. Consequently, the intermediate features are weighted and embedded into the features transmitted for executing multiple tasks at the receiver. Experiments demonstrate that the proposed model surpasses the most competitive and publicly available models by 11.4% on the CityScapes 2Task dataset and outperforms the established state-of-the-art by 3.97% on the NYU V2 3Task dataset, respectively, when the bandwidth ratio of the communication channel (i.e., compression level for transmission over the channel) is as constrained as 1 12 .

Problem

• Multi-task Semantic Communication
• Feature Representation
• Complex Inter-feature Relations

Innovation

• Graph Attention Interaction
• Multi-task Semantic Communication
• Low-bandwidth Communication