Assessment of PLSDA cross validation

TLDR

Classifying individuals by metabolic profiles is a key goal in metabolomics, but the high dimensionality relative to sample size makes it difficult, and PLSDA is a commonly used method. This paper proposes a cross‑model validation and permutation testing strategy to properly validate PLSDA classification models. The strategy involves cross‑model validation and permutation testing to guard against PLSDA overfitting, which otherwise leads to overly optimistic results. The study shows that inadequate validation yields overly optimistic performance and cautions against using PLSDA score plots for class inference.

Abstract

Classifying groups of individuals based on their metabolic profile is one of the main topics in metabolomics research. Due to the low number of individuals compared to the large number of variables, this is not an easy task. PLSDA is one of the data analysis methods used for the classification. Unfortunately this method eagerly overfits the data and rigorous validation is necessary. The validation however is far from straightforward. Is this paper we will discuss a strategy based on cross model validation and permutation testing to validate the classification models. It is also shown that too optimistic results are obtained when the validation is not done properly. Furthermore, we advocate against the use of PLSDA score plots for inference of class differences.