Reconciling theory with observations: elements of a diagnostic approach to model evaluation

TLDR

The need to reconcile theory with observations is widely recognized by initiatives such as Predictions in Ungaged Basins and NSF-sponsored Environmental Observatories. The paper proposes a diagnostic evaluation approach grounded in information theory, using signature indices to assess system process behaviors, with the goal of developing a robust Theory of Evaluation to better reconcile complex environmental models with observations. The approach employs signature indices derived from information theory, incorporates Bayesian inference for uncertainty analysis, and targets the complexity limits of models to improve predictions in ungaged basins. © 2008 John Wiley & Sons, Ltd.

Abstract

Abstract This paper discusses the need for a well‐considered approach to reconciling environmental theory with observations that has clear and compelling diagnostic power. This need is well recognized by the scientific community in the context of the ‘Predictions in Ungaged Basins’ initiative and the National Science Foundation sponsored ‘Environmental Observatories’ initiative, among others. It is suggested that many current strategies for confronting environmental process models with observational data are inadequate in the face of the highly complex and high order models becoming central to modern environmental science, and steps are proposed towards the development of a robust and powerful ‘Theory of Evaluation’. This paper presents the concept of a diagnostic evaluation approach rooted in information theory and employing the notion of signature indices that measure theoretically relevant system process behaviours. The signature‐based approach addresses the issue of degree of system complexity resolvable by a model. Further, it can be placed in the context of Bayesian inference to facilitate uncertainty analysis, and can be readily applied to the problem of process evaluation leading to improved predictions in ungaged basins. Copyright © 2008 John Wiley & Sons, Ltd.