VideoWeave: Unlocking Geometric Consistency in Video Generation via Joint Geometry-Video Modeling

2026-06-12

AI Summary

This work addresses the challenge of maintaining temporally consistent 3D geometry in large-scale video diffusion models, which often exhibit structural drift and implausible motion under viewpoint changes. To tackle this issue, the authors propose VideoWeave, an implicit geometry-guided latent-space post-training framework that encodes geometric features into implicit geometric latent variables and jointly models them with video latents in a shared denoising space, thereby imposing flexible geometric constraints on the generation distribution. By avoiding explicit geometric reconstruction, VideoWeave effectively mitigates error propagation from upstream components. The method is supported by GeoVid-80K, a newly curated paired dataset comprising 80,000 samples. Experiments demonstrate that VideoWeave significantly enhances geometric consistency in both text-to-video and image-to-video generation while preserving high visual fidelity.

Abstract

Large-scale video diffusion models often fail to preserve 3D structure over time, causing geometric drift and implausible motion under viewpoint changes. Existing methods usually enforce geometric consistency by using explicit geometry reconstructions, such as depth maps, point clouds, or reconstructed 3D structures, to define conditions, supervision, or reward signals, making the generator sensitive to errors from upstream geometry pipelines. We propose VideoWeave, a latent-space post-training framework that uses implicit geometry-model features to constrain the generative distribution, providing a more flexible and non-rigid form of guidance that mitigates the impact of reconstruction errors from geometry models. Specifically, VideoWeave adapts these features into geometry latents and jointly models them with video latents in a shared denoising space, allowing geometry to shape the generative distribution during training. To support this process, we build GeoVid-80K, an 80K-video dataset with paired appearance and geometry representations. Experiments on text-to-video and image-to-video generation show that VideoWeave improves geometric coherence while preserving strong visual quality. VideoWeave project page at https://videoweave.github.io/

Problem

• geometric consistency
• video generation
• 3D structure
• geometric drift
• viewpoint changes

Innovation

• VideoWeave
• geometric consistency
• implicit geometry
• joint modeling
• latent-space diffusion