The design of simulation studies in medical statistics

TLDR

Simulation studies use computer‑intensive procedures to evaluate statistical methods against known truths, but designing high‑quality simulations that reflect complex real‑world scenarios is challenging and rarely fully documented. This paper recommends producing a detailed protocol and provides a checklist of key considerations for designing simulation studies. The authors outline essential design elements—objectives, data‑generation procedures, simulation counts—and review current published practices. The study presents a checklist of essential design considerations for simulation studies. © 2006 John Wiley & Sons, Ltd.

Abstract

Abstract Simulation studies use computer intensive procedures to assess the performance of a variety of statistical methods in relation to a known truth. Such evaluation cannot be achieved with studies of real data alone. Designing high‐quality simulations that reflect the complex situations seen in practice, such as in prognostic factors studies, is not a simple process. Unfortunately, very few published simulation studies provide sufficient details to allow readers to understand fully all the processes required to design a simulation study. When planning a simulation study, it is recommended that a detailed protocol be produced, giving full details of how the study will be performed, analysed and reported. This paper details the important considerations necessary when designing any simulation study, including defining specific objectives of the study, determining the procedures for generating the data sets and the number of simulations to perform. A checklist highlighting the important considerations when designing a simulation study is provided. A small review of the literature identifies the current practices within published simulation studies. Copyright © 2006 John Wiley & Sons, Ltd.