Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video

2026-05-14

AI Summary

Existing methods for camera-controllable video generation rely heavily on large-scale annotated data or test-time optimization, resulting in high computational and data acquisition costs. This work proposes the Warp-as-History interface, which reformulates camera-induced warping as pseudo-historical frames aligned with the target view and injects them into the historical visual pathway of a frozen video diffusion model via a visible token selection mechanism. This approach reveals, for the first time, the model’s non-trivial zero-shot capability for camera control. With only a single camera-annotated video for lightweight LoRA fine-tuning, the method generalizes effectively to unseen scenes without architectural modifications or test-time optimization. It achieves significant improvements across multiple datasets in terms of camera trajectory adherence, visual fidelity, and motion dynamics, enabling efficient and generalizable camera-controllable video generation.

Abstract

Camera-controlled video generation has made substantial progress, enabling generated videos to follow prescribed viewpoint trajectories. However, existing methods usually learn camera-specific conditioning through camera encoders, control branches, or attention and positional-encoding modifications, which often require post-training on large-scale camera-annotated videos. Training-free alternatives avoid such post-training, but often shift the cost to test-time optimization or extra denoising-time guidance. We propose Warp-as-History, a simple interface that turns camera-induced warps into camera-warped pseudo-history with target-frame positional alignment and visible-token selection. Given a target camera trajectory, we construct camera-warped pseudo-history from past observations and feed it through the model's visual-history pathway. Crucially, we align its positional encoding with the target frames being denoised and remove warped-history tokens without valid source observations. Without any training, architectural modification, or test-time optimization, this interface reveals a non-trivial zero-shot capability of a frozen video generation model to follow camera trajectories. Moreover, lightweight offline LoRA finetuning on only one camera-annotated video further improves this capability and generalizes to unseen videos, improving camera adherence, visual quality, and motion dynamics without test-time optimization or target-video adaptation. Extensive experiments on diverse datasets confirm the effectiveness of our method.

Problem

• camera-controlled video generation
• generalizable
• zero-shot
• video generation
• camera trajectory

Innovation

• camera-controlled video generation
• zero-shot generalization
• warp-as-history
• LoRA finetuning
• positionally-aligned pseudo-history