Hyperspectral vs. RGB for Pedestrian Segmentation in Urban Driving Scenes: A Comparative Study

2025-08-15

AI Summary

RGB imaging suffers from metamerism, impairing pedestrian-background discrimination in urban driving scenarios and compromising perception safety. To address this, we investigate hyperspectral imaging (HSI) for pedestrian segmentation and propose a CSNR-JMIM-based optimal band selection method that significantly enhances spectral discriminability and suppresses false detections. By integrating principal component analysis (PCA) with CSNR-JMIM for dimensionality reduction, we conduct comparative experiments using U-Net, DeepLabV3+, and SegFormer on both pedestrian and cyclist segmentation tasks. Results demonstrate consistent performance gains: CSNR-JMIM-selected bands yield average improvements of 1.44% in IoU and 2.18% in F1-score over RGB inputs. These findings validate the efficacy and practicality of our safety-critical, HSI band optimization strategy for autonomous driving systems.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

Pedestrian segmentation in automotive perception systems faces critical safety challenges due to metamerism in RGB imaging, where pedestrians and backgrounds appear visually indistinguishable.. This study investigates the potential of hyperspectral imaging (HSI) for enhanced pedestrian segmentation in urban driving scenarios using the Hyperspectral City v2 (H-City) dataset. We compared standard RGB against two dimensionality-reduction approaches by converting 128-channel HSI data into three-channel representations: Principal Component Analysis (PCA) and optimal band selection using Contrast Signal-to-Noise Ratio with Joint Mutual Information Maximization (CSNR-JMIM). Three semantic segmentation models were evaluated: U-Net, DeepLabV3+, and SegFormer. CSNR-JMIM consistently outperformed RGB with an average improvements of 1.44% in Intersection over Union (IoU) and 2.18% in F1-score for pedestrian segmentation. Rider segmentation showed similar gains with 1.43% IoU and 2.25% F1-score improvements. These improved performance results from enhanced spectral discrimination of optimally selected HSI bands effectively reducing false positives. This study demonstrates robust pedestrian segmentation through optimal HSI band selection, showing significant potential for safety-critical automotive applications.

Problem

• Comparing hyperspectral vs RGB for pedestrian segmentation in urban driving
• Evaluating optimal HSI band selection to reduce false positives
• Assessing performance gains in pedestrian and rider segmentation metrics

Innovation

• Hyperspectral imaging enhances pedestrian segmentation
• Optimal band selection reduces false positives
• CSNR-JMIM outperforms RGB in segmentation metrics