Context-Aware Wireless Token Communication via Joint Token Masking and Detection

2026-05-03

AI Summary

This work addresses the inefficiency of conventional wireless communication systems that transmit language tokens without leveraging contextual dependencies, leading to suboptimal resource utilization under channel impairments. To overcome this limitation, the authors propose a context-aware token-level communication framework that uniquely integrates a masked language model (MLM) as a shared prior between transmitter and receiver. The transmitter selectively masks tokens deemed recoverable from context to conserve transmission power, while the receiver reconstructs the original sequence through Bayesian inference that fuses channel likelihood with the MLM-derived contextual prior. Evaluated on the Europarl and WikiText-103 datasets, the proposed method achieves up to 1.77× and 1.63× improvements in reconstruction performance, respectively, substantially outperforming existing approaches.

Abstract

The increasing use of token-based representations in language-driven applications has motivated wireless token communication, where tokens are treated as fundamental units for transmission. However, conventional communication systems overlook dependencies among tokens and allocate transmission resources uniformly, leading to inefficient use of limited wireless resources under channel impairments. In this paper, we propose a context-aware token communication framework that leverages a masked language model (MLM) as a shared contextual model between the transmitter (Tx) and receiver (Rx). At the Rx, we develop a context-aware token detection method that integrates channel likelihoods with MLM-based contextual priors under a Bayesian formulation, enabling robust token inference over noisy channels. At the Tx, we propose a context-aware token masking strategy that selectively omits tokens that can be reliably inferred at the Rx, allowing the available power budget to be concentrated on more informative tokens. These components are jointly designed through a shared MLM, establishing a unified Tx-Rx framework for efficient token transmission and detection. Simulation results demonstrate that the proposed framework significantly improves reconstruction performance compared to conventional and existing token communication schemes, achieving up to 1.77X and 1.63X performance gains on the Europarl corpus and WikiText-103 datasets, respectively.

Problem

• token communication
• context awareness
• wireless resource efficiency
• channel impairments
• token dependency

Innovation

• context-aware communication
• masked language model
• token masking
• Bayesian token detection
• wireless token transmission