Pretrained Video Models as Differentiable Physics Simulators for Urban Wind Flows

2026-03-22

AI Summary

This work proposes WinDiNet, the first approach to repurpose a large-scale pretrained video diffusion model (LTX-Video) as a differentiable physics surrogate for urban wind environment simulation. By fine-tuning on 10,000 procedurally generated building layouts with two-dimensional incompressible CFD data and incorporating a physics-informed VAE decoder loss alongside a multi-condition input mechanism, WinDiNet achieves high-fidelity prediction of wind field evolution. Built upon a 2-billion-parameter latent video Transformer architecture, the model generates 112-frame wind field sequences in under one second and supports end-to-end gradient backpropagation. This enables automatic optimization of building layouts to enhance pedestrian wind comfort and safety, with optimized designs validated against high-fidelity CFD simulations, demonstrating superior speed and accuracy compared to existing neural PDE solvers.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Designing urban spaces that provide pedestrian wind comfort and safety requires time-resolved Computational Fluid Dynamics (CFD) simulations, but their current computational cost makes extensive design exploration impractical. We introduce WinDiNet (Wind Diffusion Network), a pretrained video diffusion model that is repurposed as a fast, differentiable surrogate for this task. Starting from LTX-Video, a 2B-parameter latent video transformer, we fine-tune on 10,000 2D incompressible CFD simulations over procedurally generated building layouts. A systematic study of training regimes, conditioning mechanisms, and VAE adaptation strategies, including a physics-informed decoder loss, identifies a configuration that outperforms purpose-built neural PDE solvers. The resulting model generates full 112-frame rollouts in under a second. As the surrogate is end-to-end differentiable, it doubles as a physics simulator for gradient-based inverse optimization: given an urban footprint layout, we optimize building positions directly through backpropagation to improve wind safety as well as pedestrian wind comfort. Experiments on single- and multi-inlet layouts show that the optimizer discovers effective layouts even under challenging multi-objective configurations, with all improvements confirmed by ground-truth CFD simulations.

Problem

• urban wind comfort
• computational fluid dynamics
• design exploration
• wind safety
• CFD simulation

Innovation

• differentiable physics simulation
• video diffusion model
• urban wind comfort
• neural surrogate modeling
• gradient-based inverse design