A median absolute deviation-neural network (MAD-NN) method for atmospheric temperature data cleaning

Abstract

Some of the biggest challenges in climate change arise from bad dataset. To address this issue, we have developed a novel method for cleaning coarse atmospheric dataset; the median absolute deviation-neural network (MAD-NN) method. By combining the median absolute deviation (MAD) technique with neural network training, this method uses a sequence of steps to clean coarse atmospheric dataset and to predict high accuracy dataset for periods when measurements are not available. To demonstrate this method, we used atmospheric temperature data for 17 different observational weather stations across Nigeria. In brief:•We developed a novel method for generating consistent data stream from coarse dataset.•The MAD-NN method can be used to fill observational data gaps and remove spikes in data.•This method is specifically useful for weather observatories with coarse atmospheric data, as well as increasing the credibility of scientific findings.