Context-Aware Iterative Token Detection and Masked Transmission for Wireless Token Communication

2026-01-25

AI Summary

This work addresses the challenge of efficiently and reliably transmitting semantic tokens from language models over wireless channels. The authors propose a context-aware token communication framework that leverages a pretrained masked language model as a shared contextual prior between transmitter and receiver. The transmitter employs a context-aware masking strategy to omit highly predictable tokens, while the receiver performs Bayesian iterative detection that fuses the contextual prior with channel observations to jointly reconstruct the original semantics. This enables adaptive compression and reconstruction that are simultaneously aware of both semantic content and channel conditions. Experimental results demonstrate that the proposed method significantly improves the quality of reconstructed sentences and achieves efficient, channel-adaptive transmission rates across diverse wireless environments.

Technology Category

• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The success of large-scale language models has established tokens as compact and meaningful units for natural-language representation, which motivates token communication over wireless channels, where tokens are considered fundamental units for wireless transmission. We propose a context-aware token communication framework that uses a pretrained masked language model (MLM) as a shared contextual probability model between the transmitter (Tx) and receiver (Rx). At Rx, we develop an iterative token detection method that jointly exploits MLM-guided contextual priors and channel observations based on a Bayesian perspective. At Tx, we additionally introduce a context-aware masking strategy which skips highly predictable token transmission to reduce transmission rate. Simulation results demonstrate that the proposed framework substantially improves reconstructed sentence quality and supports effective rate adaptation under various channel conditions.

Problem

• token communication
• context-aware
• wireless transmission
• masked language model
• rate adaptation

Innovation

• context-aware token communication
• masked language model
• iterative token detection
• Bayesian inference
• adaptive rate transmission