Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History

2026-07-03

AI Summary

This work investigates how frozen vision-language-action (VLA) models represent and utilize visual history without relying on explicit memory mechanisms. Introducing, for the first time, a training-free temporal deployment auditing framework—comprising hierarchical linear probing, causal swapping interventions, history re-injection, and action prediction analysis—the study systematically compares the historical usage strategies across three VLA architectures. The findings reveal that historical information typically serves as a redundant copy of the current frame and is only invoked under severely degraded input conditions. Moreover, distinct architectures exhibit markedly different patterns of dependence on history, and the manipulability of historical information is determined by its deployment strategy rather than whether it is encoded.

Abstract

A frozen vision-language-action model (VLA) receives recent observations at every decision step, yet prior work has focused on adding memory rather than asking how existing history is represented and used. We study this temporal axis using layer-resolved linear probing and causal interchange interventions across three VLAs from two architecture families. We find a three-part dissociation. First, past-frame content remains linearly decodable throughout the network. Second, information unique to history beyond the current frame is nearly absent, indicating that stored history is largely a redundant copy of the present. Third, history is causally deployed only when the current frame is heavily degraded, while the action readout progressively loses dependence on history through the network. Although all models encode history similarly, their deployment strategies differ: under the same occlusion, one architecture increasingly relies on history as a fallback, whereas the other relies on it less. We further introduce a training-free temporal deployment audit that distinguishes these regimes. In the fallback regime, re-injecting history neither repairs occlusion nor disambiguates actions, confirming the redundancy of the stored representation. In the other regime, the same intervention reliably steers the predicted action toward the donor history. These results show that steerability depends on how history is deployed rather than whether it is encoded. VLAs do not forget the past; they largely fail to represent it as information distinct from the present. Our findings suggest that future memory augmentation should inject information unique to the past rather than simply more history.

Problem

• vision-language-action models
• visual history
• temporal representation
• memory redundancy
• history deployment

Innovation

• vision-language-action models
• temporal memory
• causal intervention
• linear probing
• history steerability