The fourth radiation transfer model intercomparison (RAMI‐IV): Proficiency testing of canopy reflectance models with ISO‐13528

TLDR

The RAMI activity evaluates the reliability of physics‑based radiative transfer models for plant canopies, a task increasingly important for satellite retrieval of terrestrial climate variables, and adopts ISO‑13528 standards to benchmark performance when true values are unknown. This study demonstrates for the first time how ISO‑13528 can be applied to proficiency testing of computer‑simulation RT models. The authors employ ISO‑13528 protocols—pre‑screening data, selecting reference solutions, and choosing proficiency statistics—to evaluate RT models, illustrated with simulation results from RAMI‑IV abstract canopy scenarios. Performance statistics of participating RT models are presented, highlighting how reference solution accuracy and tolerance criteria influence model assessment.

Abstract

The radiation transfer model intercomparison (RAMI) activity aims at assessing the reliability of physics‐based radiative transfer (RT) models under controlled experimental conditions. RAMI focuses on computer simulation models that mimic the interactions of radiation with plant canopies. These models are increasingly used in the development of satellite retrieval algorithms for terrestrial essential climate variables (ECVs). Rather than applying ad hoc performance metrics, RAMI‐IV makes use of existing ISO standards to enhance the rigor of its protocols evaluating the quality of RT models. ISO‐13528 was developed “to determine the performance of individual laboratories for specific tests or measurements.” More specifically, it aims to guarantee that measurement results fall within specified tolerance criteria from a known reference. Of particular interest to RAMI is that ISO‐13528 provides guidelines for comparisons where the true value of the target quantity is unknown. In those cases, “truth” must be replaced by a reliable “conventional reference value” to enable absolute performance tests. This contribution will show, for the first time, how the ISO‐13528 standard developed by the chemical and physical measurement communities can be applied to proficiency testing of computer simulation models. Step by step, the pre‐screening of data, the identification of reference solutions, and the choice of proficiency statistics will be discussed and illustrated with simulation results from the RAMI‐IV “abstract canopy” scenarios. Detailed performance statistics of the participating RT models will be provided and the role of the accuracy of the reference solutions as well as the choice of the tolerance criteria will be highlighted.