Dehaze-then-Splat: Generative Dehazing with Physics-Informed 3D Gaussian Splatting for Smoke-Free Novel View Synthesis

2026-04-15

AI Summary

This work addresses the challenge of poor cross-view consistency in multi-view smoke image dehazing, which leads to blurry novel-view synthesis and structural instability. The authors propose a two-stage approach: first applying generative dehazing with luminance normalization to each frame, then training a 3D Gaussian Splatting model guided by physical priors. Key innovations include physics-informed auxiliary losses—specifically depth supervision via Pearson correlation, dark channel prior regularization, and dual-source gradient matching—combined with MCMC densification and an early-stopping strategy to effectively mitigate inter-view inconsistencies. Evaluated on the Akikaze scene, the method achieves a PSNR of 20.98 dB and an SSIM of 0.683, outperforming the unregularized baseline by 1.50 dB in PSNR.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present Dehaze-then-Splat, a two-stage pipeline for multi-view smoke removal and novel view synthesis developed for Track~2 of the NTIRE 2026 3D Restoration and Reconstruction Challenge. In the first stage, we produce pseudo-clean training images via per-frame generative dehazing using Nano Banana Pro, followed by brightness normalization. In the second stage, we train 3D Gaussian Splatting (3DGS) with physics-informed auxiliary losses -- depth supervision via Pearson correlation with pseudo-depth, dark channel prior regularization, and dual-source gradient matching -- that compensate for cross-view inconsistencies inherent in frame-wise generative processing. We identify a fundamental tension in dehaze-then-reconstruct pipelines: per-image restoration quality does not guarantee multi-view consistency, and such inconsistency manifests as blurred renders and structural instability in downstream 3D reconstruction.Our analysis shows that MCMC-based densification with early stopping, combined with depth and haze-suppression priors, effectively mitigates these artifacts. On the Akikaze validation scene, our pipeline achieves 20.98\,dB PSNR and 0.683 SSIM for novel view synthesis, a +1.50\,dB improvement over the unregularized baseline.

Problem

• multi-view consistency
• image dehazing
• novel view synthesis
• 3D reconstruction
• smoke removal

Innovation

• Generative Dehazing
• 3D Gaussian Splatting
• Physics-Informed Loss
• Multi-View Consistency
• Novel View Synthesis