TS-Mask VLA: 2D Temporal-Spatial Masking for Vision-Language-Action Model with Effective Bridging

2026-07-10

AI Summary

Current vision-language-action (VLA) models suffer from limited performance on long-horizon, complex tasks due to the lack of explicit modeling of the spatiotemporal structure of action sequences and insufficient disentanglement between visual-language and action representations. To address these limitations, this work proposes the TS-Mask VLA framework, which innovatively integrates discrete diffusion-based action generation, bridging attention mechanisms, and a two-dimensional time–space masking strategy within a Transformer architecture. This design enables multi-level conditional guidance that effectively disentangles modality-specific representations while explicitly capturing both cross-temporal dependencies and inter-dimensional couplings in actions. The proposed method achieves a 95.7% average success rate on LIBERO with only 0.5B parameters and attains a state-of-the-art average sequence length of 4.19 on CALVIN, significantly outperforming larger-scale models.

Abstract

Vision-language-action (VLA) models aim to understand natural-language instructions and visual observations, and to generate and execute corresponding actions as embodied agents. Recently, autoregressive token-based action generation has driven the development of many representative VLA models. However, this paradigm often reduces action generation to next-token prediction, thereby lacking explicit modeling of the spatiotemporal structure of action sequences and the disentanglement between vision-language representations and actions, which can limit performance in long-horizon and complex scenarios. In this paper, we propose TS-Mask VLA, a vision-language-action framework for robot manipulation. TS-Mask VLA is built upon two key designs: (1) a Discrete Diffusion Action Expert equipped with a Bridge Attention conditioning bridge, which enables multi-layer conditioning from the VLM and facilitates more accurate and stable action generation; and (2) a temporal-spatial 2D masking strategy for discrete action tokens that strengthens the model's understanding of cross-time dependencies and inter-dimensional coupling, leading to more structurally consistent action sequences. We conduct extensive experiments on simulation benchmarks and real-world tasks. On LIBERO, TS-Mask VLA achieves a 95.7 percent average success rate with only 0.5B parameters, outperforming significantly larger models. On CALVIN, it attains the best average sequence length of 4.19 and strong long-horizon performance. Comprehensive analyses and ablations further validate the effectiveness of our design.

Problem

• vision-language-action
• action generation
• spatiotemporal structure
• representation disentanglement
• long-horizon tasks

Innovation

• Discrete Diffusion Action Expert
• Bridge Attention
• Temporal-Spatial 2D Masking
• Vision-Language-Action Model
• Action Sequence Modeling