A Deep Learning Approach for Pixel-level Material Classification via Hyperspectral Imaging

2024-09-20

arXiv.org

AI Summary

RGB imaging struggles with pixel-level material identification in industrial applications such as plastic waste sorting. Method: This work proposes an end-to-end classification framework integrating hyperspectral imaging (HSI) with deep learning. We introduce the first pixel-level HSI dataset for common plastics (HDPE, PET, PP, PS), leveraging Raman spectroscopy for spectral ground-truth labeling and a semi-automatic mask generation strategy. A lightweight 3D-CNN architecture is designed to jointly model spectral-spatial features. Contribution/Results: The method exhibits invariance to color, scale, and deformation, and resolves overlapping materials. It achieves 99.94% pixel-wise classification accuracy under real production-line conditions. Compared to XRF and Raman techniques, our approach offers high speed, low cost, non-destructiveness, and enhanced safety—demonstrating the feasibility and robustness of HSI+DL for industrial material identification.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Recent advancements in computer vision, particularly in detection, segmentation, and classification, have significantly impacted various domains. However, these advancements are tied to RGB-based systems, which are insufficient for applications in industries like waste sorting, pharmaceuticals, and defense, where advanced object characterization beyond shape or color is necessary. Hyperspectral (HS) imaging, capturing both spectral and spatial information, addresses these limitations and offers advantages over conventional technologies such as X-ray fluorescence and Raman spectroscopy, particularly in terms of speed, cost, and safety. This study evaluates the potential of combining HS imaging with deep learning for material characterization. The research involves: i) designing an experimental setup with HS camera, conveyor, and controlled lighting; ii) generating a multi-object dataset of various plastics (HDPE, PET, PP, PS) with semi-automated mask generation and Raman spectroscopy-based labeling; and iii) developing a deep learning model trained on HS images for pixel-level material classification. The model achieved 99.94% classification accuracy, demonstrating robustness in color, size, and shape invariance, and effectively handling material overlap. Limitations, such as challenges with black objects, are also discussed. Extending computer vision beyond RGB to HS imaging proves feasible, overcoming major limitations of traditional methods and showing strong potential for future applications.

Problem

• Classify materials at pixel level using hyperspectral imaging
• Overcome RGB limitations in industrial object characterization
• Develop deep learning model for accurate material classification

Innovation

• Combining hyperspectral imaging with deep learning
• Pixel-level material classification model
• High accuracy in diverse material conditions