Probing into Camera Control of Video Models

2026-05-14

AI Summary

Existing video generation models rely on paired data and auxiliary modules for camera control, which limits their scalability and performance in dynamic scenes, often resulting in narrow output distributions and weakened model priors. This work reframes camera control as geometric guidance and introduces a training-free displacement field mechanism that enables universal camera manipulation by applying differentiable resampling to latent features during the denoising process. For the first time, explicit geometric operations are employed as probes to systematically evaluate the camera responsiveness of mainstream video diffusion models, revealing both shared biases and key differences among them. Experiments demonstrate that the proposed approach achieves effective camera control with negligible degradation in generation quality and establishes a performance benchmark for 3D and 4D tasks.

Abstract

Video is a rich and scalable source of 3D/4D visual observations, and camera control is a key capability for video generation models to produce geometrically meaningful content. Existing approaches typically learn a mapping from camera motion to video using additional camera modules and paired data. However, such datasets are often limited in scale, diversity, and scene dynamics, which can bias the model toward a narrow output distribution and compromise the strong prior learned by the base model. These limitations motivate a different perspective on camera control. In this paper, we show that camera control need not be modeled as an implicit mapping problem, but can instead be treated as a form of geometric guidance that induces displacements across frames. Specifically, we reformulate camera control into a set of displacement fields and apply them via differentiable resampling of latent features during denoising. Our simple approach achieves effective camera control with minimal degradation across diverse quality metrics compared to fine-tuned baselines. Since our method is applicable to most video diffusion models without training, it can also serve as a probe to study the camera control capabilities of base models. Using this probe, we identify universal biases shared by representative video models, as well as disparities in their responses to camera control. Finally, we benchmark their performance in multi-view generation, offering insights into their potential for 3D/4D tasks.

Problem

• camera control
• video generation
• displacement fields
• diffusion models
• 3D/4D tasks

Innovation

• camera control
• displacement fields
• video diffusion models
• geometric guidance
• differentiable resampling