GeoStream: Toward Precise Camera Controlled Streaming Video Generation

2026-06-13

AI Summary

Existing video generation methods struggle to achieve precise metric-level control under out-of-distribution camera trajectories, and static 3D caches often fail after viewpoint changes. To address these limitations, this work proposes GeoStream, a framework that enables accurate camera control through autoregressive streaming generation. GeoStream introduces a self-refreshing 3D cache mechanism that dynamically updates geometric representations online and leverages frames generated by the model itself to construct on-policy geometric conditions for distillation training. This design aligns the training and inference distributions, effectively mitigating autoregressive drift and geometric feedback errors. Extensive quantitative and qualitative evaluations demonstrate that GeoStream significantly outperforms existing approaches in camera controllability.

Abstract

Accurate interactive camera control is essential for video-based world models, but most existing approaches learn camera motion implicitly, leading to inaccurate control under out-of-distribution trajectories. Explicit geometric conditioning improves controllability, but existing methods are non-autoregressive and rely on a static 3D cache built from an initial frame, which becomes ineffective once the viewpoint moves beyond the original frustum. We propose GeoStream, a framework that enables precise metric-scale camera control in autoregressive streaming video generation. Our method maintains a self-refreshing 3D cache that is periodically updated online from the model's own outputs: we estimate depth from the most recently generated frame, unproject to 3D, and reproject into the target view to produce point reprojections as geometric conditioning for subsequent synthesis. By the same principle, the conditioning seen during training is also rendered from the student's own generated frames, yielding a fully on-policy distillation that naturally aligns the train and inference conditioning distributions. Unlike prior work that uses off-policy condition noising, our approach trains the model against the exact error distribution it encounters at inference, mitigating both standard autoregressive drift and the second-order geometric feedback loop that arises when the cache itself is derived from generated outputs. Quantitative and qualitative results show that our approach substantially improves camera controllability.

Problem

• camera control
• video generation
• 3D cache
• autoregressive streaming
• geometric conditioning

Innovation

• autoregressive video generation
• camera control
• self-refreshing 3D cache
• on-policy distillation
• geometric conditioning