SkillComm: Skill-Driven Semantic Communication for Sequential Workflows via Incremental Token Transmission

2026-07-12

AI Summary

This work addresses the inefficiency in communication caused by repeatedly transmitting full semantic representations in ordered visual skill workflows. To overcome this limitation, we propose SkillComm, a framework that introduces reusable skill states as shared context and leverages a Skill-Book mechanism to enable on-demand updates and reuse of cross-task semantic memory, thereby departing from conventional full-transmission paradigms. Our approach integrates workflow-aware incremental token transmission, joint source-channel coding, adaptive token selection, and memory-assisted token grid reconstruction to map high-level visual intents into synchronized skill sequences. Evaluated on the Detect-Segment-Keypoint workflow using the MS COCO 2017 validation set, SkillComm reduces communication overhead by 51.2% while preserving 99.4% of normalized mean accuracy under high signal-to-noise ratios.

Abstract

As wireless visual intelligence evolves from isolated task inference to ordered skill workflows, the communication bottleneck shifts from transmitting a single semantic representation to coordinating reusable skill states under channel constraints. Existing DeepJSCC and prompt-guided visual transmitters usually treat each task as an independent full-token transmission, with limited reuse of execution memory across semantic workflows. This is inefficient for workflows such as Detect, Segment, and Keypoint, where later stages often require only state-relevant semantic updates. To this end, we propose SkillComm, a skill-driven semantic communication framework that uses reusable skill states as shared context for workflow-aware token prioritization and memory-assisted token-grid reconstruction. A shared Skill-Book maps a high-level visual intent into a synchronized executable skill sequence at the transmitter and receiver. Conditioned on this workflow, adaptive token selection exploits cross-step memory to transmit only state-active tokens through joint source-channel coding, while the receiver reconstructs a task-ready token grid by combining decoded tokens with local historical memory. Experiments on the MS COCO 2017 validation set for the Detect-Segment-Keypoint workflow show that SkillComm reduces token transmission cost by 51.2% while retaining 99.4% upper-bound-normalized average precision at high SNR. These results demonstrate that reusable skill states enable selective semantic update delivery for future agentic and embodied visual intelligence.

Problem

• semantic communication
• sequential workflows
• token transmission
• skill states
• communication efficiency

Innovation

• skill-driven semantic communication
• incremental token transmission
• reusable skill states
• workflow-aware token prioritization
• memory-assisted reconstruction