3D Smoke Scene Reconstruction Guided by Vision Priors from Multimodal Large Language Models

2026-04-07

AI Summary

This work addresses the challenges of multi-view 3D scene reconstruction under smoke interference, including scattering effects, view-dependent appearance variations, and degraded cross-view consistency. The authors propose Smoke-GS, a novel framework that, for the first time, integrates visual priors from multimodal large language models with 3D Gaussian Splatting. A lightweight, view-dependent medium-aware branch is introduced to enhance robustness and efficiency in smoky environments. Leveraging Nano-Banana-Pro for image enhancement, the method constructs an explicit 3D Gaussian representation capable of synthesizing visually clear and geometrically consistent novel views even under heavy smoke conditions. Experimental results demonstrate that Smoke-GS significantly outperforms existing approaches in both visual fidelity and reconstruction accuracy.

Abstract

Reconstructing 3D scenes from smoke-degraded multi-view images is particularly difficult because smoke introduces strong scattering effects, view-dependent appearance changes, and severe degradation of cross-view consistency. To address these issues, we propose a framework that integrates visual priors with efficient 3D scene modeling. We employ Nano-Banana-Pro to enhance smoke-degraded images and provide clearer visual observations for reconstruction and develop Smoke-GS, a medium-aware 3D Gaussian Splatting framework for smoke scene reconstruction and restoration-oriented novel view synthesis. Smoke-GS models the scene using explicit 3D Gaussians and introduces a lightweight view-dependent medium branch to capture direction-dependent appearance variations caused by smoke. Our method preserves the rendering efficiency of 3D Gaussian Splatting while improving robustness to smoke-induced degradation. Results demonstrate the effectiveness of our method for generating consistent and visually clear novel views in challenging smoke environments.

Problem

• 3D scene reconstruction
• smoke degradation
• view-dependent appearance
• cross-view consistency
• smoke scattering

Innovation

• 3D Gaussian Splatting
• smoke scene reconstruction
• vision priors
• multimodal large language models
• view-dependent appearance