TACO: Rethinking Semantic Communications with Task Adaptation and Context Embedding

2025-05-16

AI Summary

To address the challenges of receiver-side adaptability and low fidelity in semantic communication—arising from dynamic task evolution and semantic distortion—this paper proposes an end-to-end adaptable semantic encoding-decoding framework. Methodologically, it introduces (1) a novel task-adaptive modulation and context embedding co-design mechanism, enabling real-time multi-task switching at the receiver; and (2) a generative-AI-driven joint encoding-decoding architecture that integrates task-conditioned guidance with context-aware feature distillation. Evaluated on mainstream image datasets, the framework achieves a 3.2% improvement in downstream task mean Average Precision (mAP), a bandwidth compression ratio of 99.8%, and a reconstruction latency of only 12 ms. Moreover, it significantly enhances generalization capability and channel robustness, demonstrating strong performance under varying channel conditions and unseen tasks.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Recent advancements in generative artificial intelligence have introduced groundbreaking approaches to innovating next-generation semantic communication, which prioritizes conveying the meaning of a message rather than merely transmitting raw data. A fundamental challenge in semantic communication lies in accurately identifying and extracting the most critical semantic information while adapting to downstream tasks without degrading performance, particularly when the objective at the receiver may evolve over time. To enable flexible adaptation to multiple tasks at the receiver, this work introduces a novel semantic communication framework, which is capable of jointly capturing task-specific information to enhance downstream task performance and contextual information. Through rigorous experiments on popular image datasets and computer vision tasks, our framework shows promising improvement compared to existing work, including superior performance in downstream tasks, better generalizability, ultra-high bandwidth efficiency, and low reconstruction latency.

Problem

• Accurately identify and extract critical semantic information
• Adapt to evolving downstream tasks without performance degradation
• Enable flexible adaptation to multiple receiver tasks

Innovation

• Jointly captures task-specific and contextual information
• Enhances downstream task performance adaptively
• Achieves high bandwidth efficiency and low latency