Replication, Replication and Replication: Some Hard Lessons from Model Alignment

Abstract

A published simulation model (Riolo et al. 2001) was replicated in two independent implementations so that the results as well as the conceptual design align. This double replication allowed the original to be analysed and critiqued with confidence. In this case, the replication revealed some weaknesses in the original model, which otherwise might not have come to light. This shows that unreplicated simulation models and their results can not be trusted - as with other kinds of experiment, simulations need to be independently replicated.