The Latent Bridge: A Continuous Slow-Fast Channel for Real-Time Game Agents

2026-06-23

AI Summary

This work addresses the latency–quality trade-off faced by real-time game agents operating under stringent millisecond-level response constraints while attempting to leverage second-scale planning. To reconcile this tension, the authors propose the Latent Bridge architecture, which couples a frozen fast-reacting model with a slower reasoning model and trains only a lightweight, learnable bridging module. This module injects information from the slow model directly into the embedding space of the fast model via residual mappings in a continuous latent space, circumventing the computational overhead of conventional text-based bridging. Built upon frozen vision-language models (Qwen3-VL-8B-Thinking and MiniCPM-o 4.5) with LLaVA-style projections, the system achieves substantial performance gains across seven Atari games and MetaDrive, notably improving scores by 57% on MsPacman and 28% on RoadRunner, with improvements strongly correlated (r = 0.93) to the slow model’s inherent advantages.

Abstract

A real-time agent for general computer use - with games as the most demanding case - must act within tens of milliseconds while still planning over seconds. These two regimes sit at opposite ends of the latency-quality tradeoff. A reasoning VLM (Qwen3-VL-8B-Thinking) deliberates effectively but requires ~1.5 s per response - far too slow for a 15 Hz control loop. In contrast, a reactive VLM (MiniCPM-o 4.5) acts in milliseconds but underperforms on planning-heavy tasks. We couple two frozen models of matched scale (9B reactive, 8B reasoning), leaving the communication channel as the sole trainable component. The standard coupling is a Text Bridge (T): the slow model writes a suffix the fast model reads. We introduce a learned continuous Latent Bridge (L) that projects the slow model's residuals into the fast model's input-embedding space in a LLaVA-style manner, avoiding any text round-trip; both are compared against Fast-Only (F). On 7 Atari games and a driving domain (MetaDrive), tuning the action decoder per channel on held-out seeds, the Latent Bridge matches or beats the Text Bridge in every domain: it significantly improves two games (MsPacman +57%, RoadRunner +28%) and is a safe drop-in elsewhere. Combining both channels interferes destructively (RoadRunner -96%), so only one should be used. The benefit is highly predictable: the bridge helps if and only if slow reasoning already beats fast reaction (T > F) - the Latent and Text gains over Fast-Only move together at r=0.93. MetaDrive is the controlled negative, where the Latent Bridge is demonstrably inert because the Text Bridge adds no value. We release replay recordings and reproducible pipelines.

Problem

• real-time agents
• latency-quality tradeoff
• slow-fast reasoning
• continuous bridge
• planning under latency constraints

Innovation

• Latent Bridge
• slow-fast agent
• vision-language model
• real-time planning
• continuous communication channel