Vid2Fluid: 3D Dynamic Fluid Assets from Single-View Videos with Generative Gaussian Splatting

2025-03-02

AI Summary

This work addresses the problem of physics-consistent 3D dynamic fluid reconstruction from monocular video. Methodologically, it introduces the first end-to-end framework based on generative 3D Gaussian Splatting (3DGS). It estimates surface velocity via optical flow to establish a surface-to-volume mapping, incorporates a differentiable, divergence-free volumetric velocity field parameterization, and jointly optimizes fluid physical parameters (e.g., viscosity, density) alongside motion trajectories. Contributions include: (1) the first extension of generative 3DGS to monocular dynamic fluid geometry reconstruction; and (2) a surface-guided, differentiable volumetric velocity field model enabling simulation-ready asset generation and motion extrapolation editing. Experiments demonstrate high-fidelity reconstruction of gaseous, liquid, and viscous fluids. The output assets are directly importable into mainstream physics engines for simulation, significantly reducing the cost of high-quality 3D fluid modeling on publicly available videos.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

The generation of 3D content from single-view images has been extensively studied, but 3D dynamic scene generation with physical consistency from videos remains in its early stages. We propose a novel framework leveraging generative 3D Gaussian Splatting (3DGS) models to extract 3D dynamic fluid objects from single-view videos. The fluid geometry represented by 3DGS is initially generated from single-frame images, then denoised, densified, and aligned across frames. We estimate the fluid surface velocity using optical flow and compute the mainstream of the fluid to refine it. The 3D volumetric velocity field is then derived from the enclosed surface. The velocity field is then converted into a divergence-free, grid-based representation, enabling the optimization of simulation parameters through its differentiability across frames. This process results in simulation-ready fluid assets with physical dynamics closely matching those observed in the source video. Our approach is applicable to various fluid types, including gas, liquid, and viscous fluids, and allows users to edit the output geometry or extend movement durations seamlessly. Our automatic method for creating 3D dynamic fluid assets from single-view videos, easily obtainable from the internet, shows great potential for generating large-scale 3D fluid assets at a low cost.

Problem

• Generate 3D dynamic fluid objects from single-view videos.
• Estimate fluid surface velocity and refine fluid dynamics.
• Create simulation-ready fluid assets with physical consistency.

Innovation

• Generative 3D Gaussian Splatting for fluid extraction
• Optical flow for fluid surface velocity estimation
• Divergence-free grid-based velocity field optimization