Learned Image Compression for Vision-Language-Action Models

2026-06-15

AI Summary

This work addresses the challenge of real-time visual-language-action (VLA) control in bandwidth-constrained or distributed robotic systems, where high-bandwidth multi-camera inputs degrade performance and existing codecs are not optimized for downstream control tasks. The authors propose SPARC, a novel framework that introduces spatial and temporal adaptive bitrate control into VLA image compression for the first time. SPARC employs a lightweight temporal mask selector to dynamically allocate bitrates across image regions and camera views, leveraging temporal context for learned compression. Additionally, it incorporates a tilted rate-distortion loss to mitigate the suppression of rare yet critical visual patterns commonly induced by conventional entropy-based objectives. Evaluated on RoboCasa365, VLABench, and LIBERO benchmarks, SPARC significantly outperforms existing compression methods at equivalent bitrates and improves the bitrate-success trade-off in real-world teleoperation.

Abstract

Vision-language-action (VLA) models increasingly rely on high-frequency multi-camera observations, making visual communication a major bottleneck for real-time robotic control in bandwidth-constrained or distributed deployment settings. Existing image and video codecs, however, are designed to preserve generic visual fidelity rather than the control performance of downstream VLA policies. In this work, we introduce SPARC (SPatially Adaptive Rate Control), a learned image compression framework tailored for VLA-driven robots. Our key observation is that the importance of visual information varies substantially across both camera views and spatial regions within an image. Based on this observation, SPARC employs a lightweight temporal mask selector that adaptively allocates bitrate over latent representations according to task relevance while leveraging temporal context. We further introduce a tilted rate loss that stabilizes training by reducing the tendency of entropy-based objectives to over-suppress rare yet task-critical visual patterns. Experiments on diverse robotic benchmarks, including RoboCasa365, VLABench, and LIBERO, show that SPARC consistently achieves stronger control performance than conventional image/video codecs and recent learned compression methods under the same bitrate budget. We additionally demonstrate real-world deployment benefits in remote-control settings, where our method substantially improves the bitrate-success tradeoff.

Problem

• vision-language-action models
• learned image compression
• robotic control
• bandwidth-constrained communication
• control performance

Innovation

• learned image compression
• vision-language-action models
• spatially adaptive rate control
• task-aware compression
• tilted rate loss