DiffWind: Physics-Informed Differentiable Modeling of Wind-Driven Object Dynamics

2026-03-10

AI Summary

Reconstructing the dynamics of wind-driven objects from video is highly challenging due to the invisibility of wind fields, their intense spatiotemporal variability, and the complex deformations of the objects. This work proposes the first physics-informed differentiable framework that jointly optimizes spatiotemporal wind forces and object motion by integrating the Material Point Method with the Lattice Boltzmann Method. The approach employs a grid-based wind field representation and models object geometry using 3D Gaussian Splatting, enabling high-fidelity reconstruction and forward simulation through differentiable rendering and physical constraints. Evaluated on our newly introduced WD-Objects dataset, the method significantly outperforms existing techniques in both reconstruction accuracy and simulation fidelity, and further demonstrates its effectiveness and generalization capability by supporting wind field redirection and dynamic generation under novel wind conditions.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Sampling/Simulation-based Search
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Modeling wind-driven object dynamics from video observations is highly challenging due to the invisibility and spatio-temporal variability of wind, as well as the complex deformations of objects. We present DiffWind, a physics-informed differentiable framework that unifies wind-object interaction modeling, video-based reconstruction, and forward simulation. Specifically, we represent wind as a grid-based physical field and objects as particle systems derived from 3D Gaussian Splatting, with their interaction modeled by the Material Point Method (MPM). To recover wind-driven object dynamics, we introduce a reconstruction framework that jointly optimizes the spatio-temporal wind force field and object motion through differentiable rendering and simulation. To ensure physical validity, we incorporate the Lattice Boltzmann Method (LBM) as a physics-informed constraint, enforcing compliance with fluid dynamics laws. Beyond reconstruction, our method naturally supports forward simulation under novel wind conditions and enables new applications such as wind retargeting. We further introduce WD-Objects, a dataset of synthetic and real-world wind-driven scenes. Extensive experiments demonstrate that our method significantly outperforms prior dynamic scene modeling approaches in both reconstruction accuracy and simulation fidelity, opening a new avenue for video-based wind-object interaction modeling.

Problem

• wind-driven object dynamics
• video-based reconstruction
• spatio-temporal variability
• object deformation
• invisible wind

Innovation

• Differentiable Simulation
• Physics-Informed Modeling
• Wind-Object Interaction
• Material Point Method
• Lattice Boltzmann Method