Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication

2026-08-17

AI Summary

This study addresses the misalignment between existing visual token selection criteria and reconstruction quality under fixed bandwidth constraints. We propose Gated Counterfactual Rectification (GCR-C), a method that constructs candidate sets and performs full-budget counterfactual evaluations to dynamically replace baseline actions only when positive gains are confirmed. This approach effectively bridges the gap between selection strategies and final reconstruction outcomes. Experiments demonstrate that GCR-C significantly improves reconstruction quality at low-to-medium bitrates across diverse datasets and channel conditions without increasing actual bitrate consumption. Furthermore, the method exhibits robust generalization capabilities, establishing a novel paradigm for communication-aware reconstruction tasks.

Abstract

Generative visual-token communication reduces transmission load by sending only selected discrete tokens and reconstructing missing content at the receiver. However, existing token-selection criteria based on local uncertainty, importance, or diversity do not directly determine whether changing the current selection improves the final reconstruction under the same packet budget. To address this problem, we propose Gated Counterfactual Refinement for Communication (GCR-C), a rollout-style correction layer over Local-MDL. GCR-C constructs a compact diversified candidate set, evaluates each candidate through matched full-budget Local-MDL continuation, and replaces the baseline action only when a positive baseline-relative reconstruction gain is obtained. Experiments on CIFAR-10, STL-10, a coded 5G-LDPC link, and a limited high-resolution Kodak transfer show that GCR-C consistently improves reconstruction quality at active low- and medium-rate operating points without increasing the realized packet rate, while remaining effective across changes in dataset, channel condition, resolution, token grid, and tokenizer. The results also reveal a clear quality--computation tradeoff due to the additional encoder-side counterfactual evaluation.

Problem

• Visual Token Communication
• Token Selection
• Reconstruction Quality
• Packet Budget

Innovation

• Counterfactual Refinement
• Visual Token Communication
• GCR-C
• Baseline-Relative Gain
• Rollout-style Correction