Making the Black Box More Transparent: Understanding the Physical Implications of Machine Learning

TLDR

Machine learning has become popular across many domains, yet in meteorology its adoption lags because models are viewed as opaque black boxes that provide predictions without physically interpretable insights. This study aims to demystify ML black boxes by presenting a suite of model‑interpretation and visualization methods that help meteorologists understand what their models have learned. The authors apply permutation importance, feature selection, saliency and class‑activation maps, backward optimization, and novelty detection across scales to tornado, hail, winter precipitation type, and convective‑storm mode cases.

Abstract

Abstract This paper synthesizes multiple methods for machine learning (ML) model interpretation and visualization (MIV) focusing on meteorological applications. ML has recently exploded in popularity in many fields, including meteorology. Although ML has been successful in meteorology, it has not been as widely accepted, primarily due to the perception that ML models are “black boxes,” meaning the ML methods are thought to take inputs and provide outputs but not to yield physically interpretable information to the user. This paper introduces and demonstrates multiple MIV techniques for both traditional ML and deep learning, to enable meteorologists to understand what ML models have learned. We discuss permutation-based predictor importance, forward and backward selection, saliency maps, class-activation maps, backward optimization, and novelty detection. We apply these methods at multiple spatiotemporal scales to tornado, hail, winter precipitation type, and convective-storm mode. By analyzing such a wide variety of applications, we intend for this work to demystify the black box of ML, offer insight in applying MIV techniques, and serve as a MIV toolbox for meteorologists and other physical scientists.