Improving neural network generalisation

Abstract

The authors study neural network overfitting on synthetically generated and real remote sensing data. The effect of overfitting is shown by: 1) visualising the shape of the decision boundaries in feature space during the learning process, and 2) by plotting the classification accuracy of independent test sets versus the number of training cycles. A solution to the overfitting problem is proposed that, involves pre-processing the training data. The method relies on obtaining an increase of spectral coherence of individual training classes by applying k-nearest neighbour filtering. Points in feature space with class labels inconsistent with those of the majority of their neighbours are removed. This effectively simplifies the training data, and removes outliers and local inconsistencies. It is shown that using this approach can reduce the overfitting effect and increase the resulting classification accuracy.