Prediction of exchangeable potassium in soil through mid-infrared spectroscopy and deep learning: From prediction to explainability

Abstract

The ability to characterize rapidly and repeatedly exchangeable potassium (Kex) content in the soil is essential for optimizing remediation of radiocaesium contamination in agriculture. In this paper, we show how this can be now achieved using a Convolutional Neural Network (CNN) model trained on a large Mid-Infrared (MIR) soil spectral library (40,000 samples with Kex determined with 1 M NH4OAc, pH 7), compiled by the National Soil Survey Center of the United States Department of Agriculture. Using Partial Least Squares Regression as a baseline, we found that our implemented CNN leads to a significantly higher prediction performance of Kex when a large amount of data is available (10000), increasing the coefficient of determination from 0.64 to 0.79, and reducing the Mean Absolute Percentage Error from 135% to 31%. Furthermore, in order to provide end-users with required interpretive keys, we implemented the GradientShap algorithm to identify the spectral regions considered important by the model for predicting Kex. Used in the context of the implemented CNN on various Soil Taxonomy Orders, it allowed (i) to relate the important spectral features to domain knowledge and (ii) to demonstrate that including all Soil Taxonomy Orders in CNN-based modeling is beneficial as spectral features learned can be reused across different, sometimes underrepresented orders.