Using recursive feature elimination in random forest to account for correlated variables in high dimensional data

TLDR

Random forest is effective for high‑dimensional problems, but correlated predictors impair its ability to identify strong predictors, and the RF‑Recursive Feature Elimination (RF‑RFE) algorithm, which mitigates this issue in small datasets, has not yet been tested on high‑dimensional omics data. The study aims to evaluate whether RF‑RFE can handle correlated variables in high‑dimensional omics datasets. The authors integrated 202,919 genotypes and 153,422 methylation sites from 680 individuals, simulated causal genotype–methylation interactions affecting triglyceride levels, and compared the performance of RF and RF‑RFE in detecting these associations. RF‑RFE diminished the importance of both correlated and causal variables in the presence of many correlated predictors, resulting in poor detection of causal associations and suggesting that it may not scale to high‑dimensional data.

Abstract

Random forest (RF) is a machine-learning method that generally works well with high-dimensional problems and allows for nonlinear relationships between predictors; however, the presence of correlated predictors has been shown to impact its ability to identify strong predictors. The Random Forest-Recursive Feature Elimination algorithm (RF-RFE) mitigates this problem in smaller data sets, but this approach has not been tested in high-dimensional omics data sets. We integrated 202,919 genotypes and 153,422 methylation sites in 680 individuals, and compared the abilities of RF and RF-RFE to detect simulated causal associations, which included simulated genotype–methylation interactions, between these variables and triglyceride levels. Results show that RF was able to identify strong causal variables with a few highly correlated variables, but it did not detect other causal variables. Although RF-RFE decreased the importance of correlated variables, in the presence of many correlated variables, it also decreased the importance of causal variables, making both hard to detect. These findings suggest that RF-RFE may not scale to high-dimensional data.