British Journal of Mathematical and Statistical Psychology · 1992 · 134 citations · 0 references
Natural SciencesBiostatisticsExperimental DesignStatisticsStochastic ModelingInferential StatisticsMonte Carlo MethodsMonte Carlo SimulationRobust StatisticsSimulation MethodsStatistical InferenceMonte Carlo AnalysisMonte Carlo Simulation ModelingRobustness AssessmentSimulation StudiesStatistical Conclusion ValidityStatistical DiagnosticsStatistical ValidityType Ii Error
Simulation studies frequently omit a rationale for the number of Monte Carlo iterations, fail to define robustness, and use insufficient iterations, compromising control of Type I and Type II errors. This paper applies a well‑known procedure to objectively determine the exact number of iterations required to confidently detect departures from robustness in Monte Carlo results. A table lists the number of iterations needed to detect departures from a series of nominal Type I error rates.
A recent survey of simulation studies concluded that an overwhelming majority of papers do not report a rationale for the decision regarding the number of Monte Carlo iterations. A surprisingly large number of reports do not contain a justifiable definition of robustness and many studies are conducted with an insufficient number of iterations to achieve satisfactory statistical conclusion validity. The implication is that we do not follow our own advice regarding the management of Type I and Type II errors when conducting Monte Carlo experiments. This paper reports a straightforward application of a well‐known procedure for the purpose of objectively determining the exact number of iterations necessary to confidently detect departures from robustness in Monte Carlo results. A table of the number of iterations necessary to detect departures from a series of nominal Type I error rates is included.