OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Control

2026-04-07

AI Summary

Existing video generation models struggle to disentangle scene content from camera motion, limiting creative flexibility. This work proposes OmniCamera, a unified framework that explicitly decouples and independently controls dynamic scene content and arbitrary camera trajectories. To achieve this, we construct OmniCAM, a hybrid dataset combining real-world and synthetic data, and introduce a dual-level curriculum co-training strategy—progressing from simple to complex conditions at the conditioning level and from synthetic to real data at the data level—alongside a multimodal conditional control architecture to effectively mitigate modality conflicts. The proposed method enables flexible control over complex camera motions while preserving high visual fidelity, achieving state-of-the-art performance across multiple video generation tasks.

Abstract

Video fundamentally intertwines two crucial axes: the dynamic content of a scene and the camera motion through which it is observed. However, existing generation models often entangle these factors, limiting independent control. In this work, we introduce OmniCamera, a unified framework designed to explicitly disentangle and command these two dimensions. This compositional approach enables flexible video generation by allowing arbitrary pairings of camera and content conditions, unlocking unprecedented creative control. To overcome the fundamental challenges of modality conflict and data scarcity inherent in such a system, we present two key innovations. First, we construct OmniCAM, a novel hybrid dataset combining curated real-world videos with synthetic data that provides diverse paired examples for robust multi-task learning. Second, we propose a Dual-level Curriculum Co-Training strategy that mitigates modality interference and synergistically learns from diverse data sources. This strategy operates on two levels: first, it progressively introduces control modalities by difficulties (condition-level), and second, trains for precise control on synthetic data before adapting to real data for photorealism (data-level). As a result, OmniCamera achieves state-of-the-art performance, enabling flexible control for complex camera movements while maintaining superior visual quality.

Problem

• video generation
• camera control
• content disentanglement
• multi-task learning
• modality conflict

Innovation

• camera-content disentanglement
• multi-task video generation
• hybrid dataset
• curriculum co-training
• arbitrary camera control