POC plots: calibrating species distribution models with presence‐only data

TLDR

Statistical models are widely used to predict species distributions, but their performance is usually assessed only by discrimination metrics such as AUC, which ignore calibration. This study introduces the presence‑only calibration (POC) plot, a tool for measuring model calibration using only presence data. The POC plot generalizes Hirzel et al.’s predicted/expected curves to presence‑only data, enabling visual assessment of whether predictions are proportional to conditional probability of presence. Using POC plots, we recalibrated DOMAIN models for 226 species across six regions, significantly enhancing their predictive performance.

Abstract

Statistical models are widely used for predicting species' geographic distributions and for analyzing species' responses to climatic and other predictor variables. Their predictive performance can be characterized in two complementary ways: discrimination, the ability to distinguish between occupied and unoccupied sites, and calibration, the extent to which a model correctly predicts conditional probability of presence. The most common measures of model performance, such as the area under the receiver operating characteristic curve (AUC), measure only discrimination. In contrast, we introduce a new tool for measuring model calibration: the presence‐only calibration plot, or POC plot. This tool relies on presence‐only evaluation data, which are more widely available than presence–absence evaluation data, to determine whether predictions are proportional to conditional probability of presence. We generalize the predicted/expected curves of Hirzel et al. to produce a presence‐only analogue of traditional (presence–absence) calibration curves. POC plots facilitate visual exploration of model calibration, and can be used to recalibrate badly calibrated models. We demonstrate their use by recalibrating models made by the DOMAIN modeling method on a comprehensive set of 226 species from six regions of the world, significantly improving DOMAIN's predictive performance.