Validation of imprecise probability models

Scott Ferson, William L. Oberkampf

International Journal of Reliability and Safety · 2009 · 106 citations · 39 references

Concepts

TL;DR

Validation assesses how well a model’s predictions match empirical observations, a task that becomes complex when predictions or data are expressed as uncertain numbers such as intervals, probability distributions, p‑boxes, or other structures. The authors propose a unified validation measure that computes the shortest possible distance between the uncertain prediction and data distributions, thereby characterizing both shape discrepancies and differences between realizations under imprecise probability.

Abstract

Validation is the assessment of the match between a model's predictions and empirical observations. It can be complex when either data or the prediction is characterised as an uncertain number (i.e. interval, probability distribution, p-box, or more general structure). Validation could measure the discrepancy between the shapes of the two uncertain numbers representing prediction and data, or it could characterise the differences between realisations drawn from the respective uncertain numbers. The unification between these two concepts relies on defining the validation measure between prediction and data as the shortest possible distance given the imprecision about the distributions and their dependencies.

References

39