Mixture of Semantics Transmission for Generative AI-Enabled Semantic Communication Systems

2025-09-11

AI Summary

To address the low semantic transmission efficiency and the trade-off between channel resource utilization and reconstruction quality in generative-AI-driven wireless semantic communication, this paper proposes a Mixture-of-Semantics (MoS) transmission strategy. MoS leverages semantic segmentation to precisely distinguish Regions of Interest (ROI) from Regions of Non-Interest (RONI), enabling differentiated semantic encoding and transmission: ROI encoding prioritizes structural fidelity, while RONI encoding emphasizes semantic consistency. At the receiver, a diffusion model jointly reconstructs the full image. The framework integrates semantic segmentation, rate-distortion-optimized coding, and generative reconstruction to realize end-to-end semantic-level communication. Experiments demonstrate that MoS achieves a 3.2 dB gain in ROI peak signal-to-noise ratio (PSNR) and an 18.7% improvement in RONI CLIP similarity over baseline methods, marking the first approach in semantic communication to simultaneously optimize visual fidelity and semantic relevance.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Mixture of Experts (MoE)
• Multiagent Systems: Agent Communication

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In this paper, we propose a mixture of semantics (MoS) transmission strategy for wireless semantic communication systems based on generative artificial intelligence (AI). At the transmitter, we divide an image into regions of interest (ROI) and reigons of non-interest (RONI) to extract their semantic information respectively. Semantic information of ROI can be allocated more bandwidth, while RONI can be represented in a compact form for transmission. At the receiver, a diffusion model reconstructs the full image using the received semantic information of ROI and RONI. Compared to existing generative AI-based methods, MoS enables more efficient use of channel resources by balancing visual fidelity and semantic relevance. Experimental results demonstrate that appropriate ROI-RONI allocation is critical. The MoS achieves notable performance gains in peak signal-to-noise ratio (PSNR) of ROI and CLIP score of RONI.

Problem

• Optimizing bandwidth allocation for ROI and RONI transmission
• Balancing visual fidelity and semantic relevance in communication
• Reconstructing full images using generative AI with semantic data

Innovation

• Divides images into ROI and RONI regions
• Allocates more bandwidth to ROI semantics
• Uses diffusion model to reconstruct images