Subliminal Clocks: Latent Time Modelling in Diffusion Language Models

2026-07-02

AI Summary

This study investigates whether diffusion language models implicitly encode temporal information about the denoising process within their internal representations and how this affects generation behavior. Through linear probing and geometric analysis of activation spaces, the work provides the first evidence that an interpretable, decodable implicit time signal exists in the model’s residual stream. Building upon this finding, the authors construct a low-dimensional subspace to intervene on this signal, enabling systematic control over the model’s perceived progress in the denoising trajectory. Experiments demonstrate that this temporal representation is stable across multiple layers, exhibits clear structure, and supports controllable modulation of generation confidence and entropy, thereby offering a principled mechanism for steering generative dynamics.

Abstract

Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive models. Unlike standard diffusion-based approaches, DLMs are not explicitly conditioned on a timestep, raising a natural question: do these models internally represent denoising progress, and how is such information used downstream? In this work, we show that DLMs do in fact encode a latent representation related to the diffusion timestep within their residual streams. We find that this signal can be reliably extracted using probes across layers, indicating that denoising progress is decodable from internal activations. We further demonstrate that steering the model along a low-dimensional subspace associated with the inferred timestep allows us to systematically modulate its notion of denoising progress, leading to predictable changes in model confidence and entropy. Finally, we analyse the geometry of the identified representation, showing that it exhibits structured and interpretable properties in activation space, and shedding light on how such a signal is processed by these models.

Problem

• Diffusion Language Models
• latent time
• denoising progress
• timestep representation
• internal activations

Innovation

• latent time representation
• diffusion language models
• probing
• denoising progress
• activation geometry