A Time Series Dataset of NIR Spectra and RGB and NIR-HSI Images of the Barley Germination Process

2025-04-23

AI Summary

This work addresses the lack of high-quality multimodal time-series data for dynamic monitoring of barley germination. We introduce the first publicly available, multi-temporal, multispectral barley germination benchmark dataset, featuring pixel-level segmentation masks and per-grain germination labels. It comprises RGB images, NIR hyperspectral images (with corresponding masks), and near-infrared spectra—acquired every 24 hours over five days for 2,242 individual barley seeds—and synchronized with expert-annotated germination status. Grain segmentation under black-background conditions is performed automatically using Otsu’s thresholding, followed by manual verification based on standardized physiological criteria. The dataset enables cross-modal temporal modeling integrating RGB, NIR spectral, and NIR-HSI modalities, thereby significantly advancing non-destructive phenotyping, germination process modeling, and the development of multimodal time-series machine learning methods.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

We provide an open-source dataset of RGB and NIR-HSI (near-infrared hyperspectral imaging) images with associated segmentation masks and NIR spectra of 2242 individual malting barley kernels. We imaged every kernel pre-exposure to moisture and every 24 hours after exposure to moisture for five consecutive days. Every barley kernel was labeled as germinated or not germinated during each image acquisition. The barley kernels were imaged with black filter paper as the background, facilitating straight-forward intensity threshold-based segmentation, e.g., by Otsu's method. This dataset facilitates time series analysis of germination time for barley kernels using either RGB image analysis, NIR spectral analysis, NIR-HSI analysis, or a combination hereof.

Problem

• Track barley germination using RGB and NIR-HSI images
• Analyze time series data of individual barley kernels
• Enable segmentation and spectral analysis of germination stages

Innovation

• Uses RGB and NIR-HSI imaging for barley germination
• Applies Otsu's method for segmentation
• Combines time series with spectral analysis