A RATIONAL APPROACH FOR DETERMINING THE APPROPRIATE NUMBER OF MONTE CARLO SIMULATIONS REQUIRED IN HEALTH ECONOMIC ANALYSES

Purpose:    Monte Carlo simulation is a commonly used method to account for uncertainty in cost-effectiveness analysis - by convention the number of simulations used is an arbitrary round number e.g. 1,000. We present a rational approach for determining the appropriate number of simulations to use in any particular simulation, along with a pilot of this method.

Method:    First the analyst should specify: (a) what is the purpose of the analysis; (b) how accurate must the results be; and (c) how many confirmatory simulations should be performed once the output appears to stabilise.    The most straightforward means of implementing this approach is through iteration. Eventually additional simulations will have no substantial impact on the output of interest – at this point confirmatory simulations must be performed to confirm that the stabilisation is permanent rather than by random chance.    This method was then piloted on a recently published cost-effectiveness analysis of screening for post-natal depression in UK primary care. The number of simulations required to derive either the ICERs or the net benefit associated with each of the 12 strategies was retrospectively calculated to varying degrees of accuracy.

Result:    The original analysis over 10,000 simulations found three strategies to be dominated and one to be extendedly dominated, resulting in seven ICERs. Each ICER appeared to stabilise to the nearest £1,000 after 1,122 simulations. The ICERs failed to permanently stabilise to the nearest £100 or £10 within the 10,000 simulations available.    Net benefit associated with each strategy permanently stabilised to the nearest £100 in 50 simulations, and to the nearest £10 after 5,392 simulations. The net benefit failed to permanently stabilise to the nearest £1 within the 10,000 simulations available.    Work is currently being undertaken to derive further simulations from the model in order to investigate these results further.

Conclusion:    The appropriate number of Monte Carlo simulations varies according to the purpose of the analysis, and is highly dependent upon the level of accuracy required. Adopting a conventional but arbitrary number of simulations generally results in either wasted computing time or spurious precision, as evidenced by the pilot study. We have introduced a simple and rational framework for calculating the appropriate number of Monte Carlo simulations, which could be adopted in future cost-effectiveness analyses.