Spectroscopy Analysis with Machine Learning Regression for the Quantification of Carbon and Nitrogen Contents in Inceptisol and Oxisol Soil Types: Comparing Different Preprocessing and Validation methods as well as Feature Importance

2026-07-01

AI Summary

Traditional methods for soil carbon and nitrogen analysis are time-consuming, costly, and destructive, making them unsuitable for the rapid, non-destructive requirements of modern agriculture. This study addresses this limitation by integrating near-infrared spectroscopy with machine learning to develop an innovative stacking ensemble model tailored for Inceptisols and Oxisols. The proposed approach combines Savitzky–Golay filtering, NIPALS-Huber robust outlier removal, Kennard–Stone sample partitioning, and base learners including PLS, SVR, and Ridge regression, fused via a linear meta-learner. It achieves stable predictive performance with RPD > 2.0 and minimal overfitting across both soil types, significantly outperforming conventional techniques. Furthermore, the work elucidates how soil type influences model generalizability, offering a reliable foundation for in-field, rapid decision-making in precision agriculture.

Abstract

Near-Infrared (NIR) spectroscopy has emerged as a promising alternative to traditional soil analysis methods, offering advantages such as speed, low cost, and non-destructive testing. This work proposes a machine learning (ML) approach to calibrate predictive models for carbon (C) and nitrogen (N) content in Oxisols and Inceptisols, utilizing NIR spectral data acquired with a portable MyNIR device. Various preprocessing methods were evaluated, with the most effective being the Savitzky-Golay (SG) filter and a robust outlier removal method based on the Nonlinear Iterative Partial Least Squares (NIPALS) algorithm coupled with a Huber loss function. Multiple validation strategies were compared, including 10-fold cross-validation, leave-one-out, and holdout via the Kennard-Stone method, followed by standardization. Stacking ensemble learning models were employed, using Partial Least Squares (PLS), Support Vector Regression (SVR), and Ridge as base models, with linear regression as the meta-model. The models were evaluated using R2, Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Ratio of Performance Deviation (RPD) metrics. The performance gap between soil types suggests the influence of pedological characteristics. Furthermore, the models achieved an RPD > 2.0 with low overfitting, validating the potential of this approach for rapid C and N quantification. This study contributes to the optimization of sustainable agricultural practices, aligning with the demand for efficient and environmentally friendly analytical methods. The developed technique enables faster decision-making for producers and consultants based on organic matter content, fertility indicators, and nutrient availability.

Problem

• Near-Infrared Spectroscopy
• Carbon Quantification
• Nitrogen Quantification
• Soil Analysis
• Machine Learning Regression

Innovation

• Near-Infrared Spectroscopy
• Machine Learning Regression
• Stacking Ensemble
• Soil Carbon and Nitrogen Quantification
• Robust Outlier Removal