Improving the quality evaluation process of machine learning algorithms applied to landslide time series analysis

Abstract

The introduction of Machine Learning (ML) in the geotechnical community has led to numerous applications for monitoring data elaboration. These techniques demonstrate promising performance in comparison to conventional methods aimed at determining the future behavior of a landslide. In this context, it is fundamental to have access to reliable methodologies and procedures to assess the quality of algorithms' predictions. This article proposes an improved method for evaluating ML algorithms applied to landslide time series analysis. The method relies on modified metrics that are sensible to biased classification due to imbalanced datasets, also enabling the evaluation of both regression and classification models using the same criteria. The calculated metrics include Accuracy, Precision, Recall, and F1-Score, each one representing a different aspect of the forecasting model effectiveness. Results obtained from the application of the proposed method to datasets collected by automated monitoring systems proved to be informative of the performance of the model and provides the means for objective comparison with other forecasting algorithms, making it a valuable tool to improve the prediction process reliability. In particular, the custom metrics allowed for a better evaluation of algorithms skewed in favor of the dominant class/classes, which are common occurrence in landslide displacement datasets. In these cases, the proposed approach highlighted the inability of the forecasting model in predicting critical events, presenting a more accurate representation of its performances compared to results obtained with standard approaches.