Making Models Match: Replicating an Agent-Based Model

Abstract

Scientists have increasingly employed computer models in their work. Recent years have seen a
\nproliferation of agent-based models in the natural and social sciences. But with the exception of
\na few "classic" models, most of these models have never been replicated by anyone but the
\noriginal developer. As replication is a critical component of the scientific method and a core
\npractice of scientists, we argue herein for an increased practice of replication in the agent-
\nbased modeling community, and for widespread discussion of the issues surrounding
\nreplication. We begin by clarifying the concept of replication as it applies to ABM. Furthermore
\nwe argue that replication may have even greater benefits when applied to computational models
\nthan when applied to physical experiments. Replication of computational models affects model
\nverification and validation and fosters shared understanding about modeling decisions. To
\nfacilitate replication, we must create standards for both how to replicate models and how to
\nevaluate the replication. In this paper, we present a case study of our own attempt to replicate a
\nclassic agent-based model. We begin by describing an agent-based model from political
\nscience that was developed by Axelrod and Hammond. We then detail our effort to replicate that
\nmodel and the challenges that arose in recreating the model and in determining if the
\nreplication was successful. We conclude this paper by discussing issues for (1) researchers
\nattempting to replicate models and (2) researchers developing models in order to facilitate the
\nreplication of their results.