Summarizing multiple aspects of model performance in a single diagram

TLDR

The diagram is a general tool particularly useful for evaluating complex geophysical models. The authors propose methods to display statistical significance, account for observational uncertainty and internal variability, and use the diagram’s geometry to inform skill‑score weighting. The diagram concisely summarizes pattern correlations, RMS differences, and variance ratios, enabling comparison of model merits and monitoring performance changes.

Abstract

A diagram has been devised that can provide a concise statistical summary of how well patterns match each other in terms of their correlation, their root‐mean‐square difference, and the ratio of their variances. Although the form of this diagram is general, it is especially useful in evaluating complex models, such as those used to study geophysical phenomena. Examples are given showing that the diagram can be used to summarize the relative merits of a collection of different models or to track changes in performance of a model as it is modified. Methods are suggested for indicating on these diagrams the statistical significance of apparent differences and the degree to which observational uncertainty and unforced internal variability limit the expected agreement between model‐simulated and observed behaviors. The geometric relationship between the statistics plotted on the diagram also provides some guidance for devising skill scores that appropriately weight among the various measures of pattern correspondence.