GimbalDiffusion: Gravity-Aware Camera Control for Video Generation

2025-12-09

AI Summary

Existing text-to-video generation methods struggle to achieve fine-grained control over camera motion and orientation, primarily due to reliance on relative or ambiguous trajectory representations lacking explicit geometric constraints. To address this, we propose a gravity-anchored absolute camera modeling framework: (1) a novel gravity-aligned Euler angle parameterization that encodes camera pose in an absolute world coordinate system; (2) a zero-pitch condition labeling strategy to decouple conflicting textual semantics from camera instructions; and (3) SpatialVID-HQ-Rebalanced—a new evaluation benchmark supporting large pitch angles. Built upon diffusion models, our method integrates panoramic 360° trajectory synthesis with null-pitch conditioning. Experiments demonstrate substantial improvements in camera trajectory controllability, generation consistency, and geometric plausibility, outperforming state-of-the-art methods on SpatialVID-HQ-Rebalanced.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent progress in text-to-video generation has achieved remarkable realism, yet fine-grained control over camera motion and orientation remains elusive. Existing approaches typically encode camera trajectories through relative or ambiguous representations, limiting explicit geometric control. We introduce GimbalDiffusion, a framework that enables camera control grounded in physical-world coordinates, using gravity as a global reference. Instead of describing motion relative to previous frames, our method defines camera trajectories in an absolute coordinate system, allowing precise and interpretable control over camera parameters without requiring an initial reference frame. We leverage panoramic 360-degree videos to construct a wide variety of camera trajectories, well beyond the predominantly straight, forward-facing trajectories seen in conventional video data. To further enhance camera guidance, we introduce null-pitch conditioning, an annotation strategy that reduces the model's reliance on text content when conflicting with camera specifications (e.g., generating grass while the camera points towards the sky). Finally, we establish a benchmark for camera-aware video generation by rebalancing SpatialVID-HQ for comprehensive evaluation under wide camera pitch variation. Together, these contributions advance the controllability and robustness of text-to-video models, enabling precise, gravity-aligned camera manipulation within generative frameworks.

Problem

• Enables precise camera control in video generation using physical coordinates.
• Overcomes limitations of ambiguous camera motion representations in existing methods.
• Enhances controllability with gravity-aligned camera manipulation and reduced text reliance.

Innovation

• Gravity-aware absolute coordinate camera control
• Panoramic video data for diverse trajectory training
• Null-pitch conditioning to reduce text-camera conflict