Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations

TLDR

Watershed models simulate watershed processes, but no comprehensive guidance exists for evaluating their accuracy against measured flow and constituent data. This study develops guidelines for evaluating watershed model accuracy by recommending evaluation techniques, reviewing performance ranges, and establishing rating criteria for streamflow and sediment/nutrient transport. The authors performed a thorough literature review of model evaluation methods, defined performance ratings for key statistics, discussed additional considerations, and illustrated the guidelines with a case study. The guidelines recommend using NSE, PBIAS, and RSR along with graphical techniques, with satisfactory thresholds of NSE > 0.50, RSR < 0.70, and PBIAS + 25 % for streamflow, + 55 % for sediment, and + 70 % for N and P, while also accounting for data quality, calibration, time step, and project scope.

Abstract

Watershed models are powerful tools for simulating the effect of watershed processes and management on soil and water resources. However, no comprehensive guidance is available to facilitate model evaluation in terms of the accuracy of simulated data compared to measured flow and constituent values. Thus, the objectives of this research were to: (1) determine recommended model evaluation techniques (statistical and graphical), (2) review reported ranges of values and corresponding performance ratings for the recommended statistics, and (3) establish guidelines for model evaluation based on the review results and project-specific considerations; all of these objectives focus on simulation of streamflow and transport of sediment and nutrients. These objectives were achieved with a thorough review of relevant literature on model application and recommended model evaluation methods. Based on this analysis, we recommend that three quantitative statistics, Nash-Sutcliffe efficiency (NSE), percent bias (PBIAS), and ratio of the root mean square error to the standard deviation of measured data (RSR), in addition to the graphical techniques, be used in model evaluation. The following model evaluation performance ratings were established for each recommended statistic. In general, model simulation can be judged as satisfactory if NSE > 0.50 and RSR < 0.70, and if PBIAS + 25% for streamflow, PBIAS + 55% for sediment, and PBIAS + 70% for N and P. For PBIAS, constituent-specific performance ratings were determined based on uncertainty of measured data. Additional considerations related to model evaluation guidelines are also discussed. These considerations include: single-event simulation, quality and quantity of measured data, model calibration procedure, evaluation time step, and project scope and magnitude. A case study illustrating the application of the model evaluation guidelines is also provided.