Comparison of feature importance measures as explanations for classification models

TLDR

Explainable AI is an emerging research area that seeks to help users understand model behavior, with feature importance being the most popular explanation technique, yet multiple approaches exist for measuring it, notably global and local methods. This study aims to compare different feature importance measures across linear and non‑linear models and local interpretable model‑agnostic explanations. The authors applied logistic regression with L1 regularization, random forests, and LIME explanations to the UCI breast cancer dataset and a newly collected running injury dataset. Results indicate that the most important features vary by technique, suggesting that combining multiple explanation methods—especially using local explanations for critical cases like false negatives—yields more reliable and trustworthy insights.

Abstract

Abstract Explainable artificial intelligence is an emerging research direction helping the user or developer of machine learning models understand why models behave the way they do. The most popular explanation technique is feature importance. However, there are several different approaches how feature importances are being measured, most notably global and local. In this study we compare different feature importance measures using both linear (logistic regression with L1 penalization) and non-linear (random forest) methods and local interpretable model-agnostic explanations on top of them. These methods are applied to two datasets from the medical domain, the openly available breast cancer data from the UCI Archive and a recently collected running injury data. Our results show that the most important features differ depending on the technique. We argue that a combination of several explanation techniques could provide more reliable and trustworthy results. In particular, local explanations should be used in the most critical cases such as false negatives.