SmokeSeer: 3D Gaussian Splatting for Smoke Removal and Scene Reconstruction

2025-09-21

AI Summary

Dynamic, multi-density smoke in real-world scenarios severely degrades image quality and impedes 3D perception. To address this, we propose the first multi-view video-based joint reconstruction and smoke removal method explicitly designed for smoke penetration. Our approach introduces 3D Gaussian splatting into smoke modeling—enabling explicit decoupling of smoke and scene components for the first time. By fusing cross-modal thermal infrared and RGB observations, we jointly optimize scene geometry, appearance, and smoke distribution to mitigate scattering effects under dynamic smoke conditions, achieving high-fidelity 3D reconstruction and image restoration. We validate our method on both synthetic and real-world datasets. Furthermore, we construct and publicly release the first synchronized multi-view smoke dataset featuring co-registered thermal-RGB video sequences, alongside all source code and preprocessing tools.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Smoke in real-world scenes can severely degrade the quality of images and hamper visibility. Recent methods for image restoration either rely on data-driven priors that are susceptible to hallucinations, or are limited to static low-density smoke. We introduce SmokeSeer, a method for simultaneous 3D scene reconstruction and smoke removal from a video capturing multiple views of a scene. Our method uses thermal and RGB images, leveraging the fact that the reduced scattering in thermal images enables us to see through the smoke. We build upon 3D Gaussian splatting to fuse information from the two image modalities, and decompose the scene explicitly into smoke and non-smoke components. Unlike prior approaches, SmokeSeer handles a broad range of smoke densities and can adapt to temporally varying smoke. We validate our approach on synthetic data and introduce a real-world multi-view smoke dataset with RGB and thermal images. We provide open-source code and data at the project website.

Problem

• Removing smoke from images to improve visibility
• Reconstructing 3D scenes from videos with smoke
• Handling varying smoke densities and dynamic smoke

Innovation

• Fuses thermal and RGB images
• Uses 3D Gaussian splatting
• Decomposes scene into smoke components