Hyperspectral Smoke Segmentation via Mixture of Prototypes

2026-02-11

AI Summary

Visible-light-based methods struggle to accurately segment smoke regions obscured by clouds or exhibiting semi-transparency due to limited spectral information. To address this challenge, this work introduces HSSDataset—the first hyperspectral smoke segmentation dataset—and proposes a Hybrid Prototype Network. The network mitigates spectral interference through band decoupling, models smoke spectral patterns via prototype learning, and incorporates a dual-level adaptive routing mechanism to enable joint spatial-spectral weighting. Evaluated on both hyperspectral and multispectral (RGB-infrared) modalities, the proposed method significantly outperforms existing approaches, establishing a new paradigm for spectral-based smoke segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Smoke segmentation is critical for wildfire management and industrial safety applications. Traditional visible-light-based methods face limitations due to insufficient spectral information, particularly struggling with cloud interference and semi-transparent smoke regions. To address these challenges, we introduce hyperspectral imaging for smoke segmentation and present the first hyperspectral smoke segmentation dataset (HSSDataset) with carefully annotated samples collected from over 18,000 frames across 20 real-world scenarios using a Many-to-One annotations protocol. However, different spectral bands exhibit varying discriminative capabilities across spatial regions, necessitating adaptive band weighting strategies. We decompose this into three technical challenges: spectral interaction contamination, limited spectral pattern modeling, and complex weighting router problems. We propose a mixture of prototypes (MoP) network with: (1) Band split for spectral isolation, (2) Prototype-based spectral representation for diverse patterns, and (3) Dual-level router for adaptive spatial-aware band weighting. We further construct a multispectral dataset (MSSDataset) with RGB-infrared images. Extensive experiments validate superior performance across both hyperspectral and multispectral modalities, establishing a new paradigm for spectral-based smoke segmentation.

Problem

• hyperspectral imaging
• smoke segmentation
• spectral information
• cloud interference
• semi-transparent smoke

Innovation

• hyperspectral imaging
• smoke segmentation
• mixture of prototypes
• adaptive band weighting
• spectral representation