Timesteps of Mamba Align with Human Reading Times

2026-06-29

AI Summary

This study investigates the alignment between internal processing dynamics in language models and human reading behavior. By analyzing the per-token dynamic discretization time step Δₜ in the Mamba state-space model and comparing it with naturalistic eye-tracking data from human readers, the authors find that Δₜ significantly predicts human reading times. This predictive relationship remains robust even after controlling for established psycholinguistic predictors such as GPT-2 surprisal. The results suggest that Mamba’s continuous-time memory mechanism effectively captures temporal characteristics of human language comprehension, offering a novel computational framework for modeling memory updating and information integration in human language processing.

Abstract

This study demonstrates an alignment of per-word processing time in a popular state-space language model Mamba and human readers. In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the discretization timestep $Δ_t$, determined dynamically in response to the input. Using a naturalistic reading dataset, we show that the per-word timestep from Mamba is a significant predictor of human reading times, and remains significant even when known predictors such as GPT-2 surprisal are controlled for. We further suggest, through formal analysis of Mamba's architecture and internal dynamics, that Mamba can serve as a new, valuable lens to look at human real-time language processing with ever-updated memory, because it allows us to look at how each module (layer) weighs short- and long-term information retention, and how noise may interact with dynamic, continuous memory representation. Code is available online.

Problem

• Mamba
• reading times
• state-space model
• language processing
• timestep

Innovation

• Mamba
• reading time
• state-space model
• dynamic timestep
• language processing