To Register      SMDM Homepage

Tuesday, 19 October 2004

This presentation is part of: Poster Session - Clinical Strategies; Judgment and Decison Making

VARIANCE REDUCTION WHEN COMPARING POLICIES IN COHORT SIMULATIONS

Steven M. Shechter, MS, University of Pittsburgh, Industrial Engineering, Pittsburgh, PA, R. Scott Braithwaite, MD, University of Pittsburgh, Section of Decision Sciences and Clinical Systems Modeling, Pittsburgh, PA, Andrew Schaefer, PhD, University of Pittsburgh, Department of Industrial Engineering, Pittsburgh, PA, and Mark S. Roberts, MD, MPP, University of Pittsburgh, Section of Decision Sciences and Clinical Systems Managment, Pittsburgh, PA.

Purpose:

When using Monte Carlo cohort simulation to compare outcomes between different policies, one desires a tight confidence interval (CI) around the difference in sample means for a fixed 100(1-α)% CI. One obvious way to achieve that is to increase the number of replications. The purpose of this study is to demonstrate how the variance reduction technique of common random numbers (CRN) may tighten the CI without increasing the replications.

Methods:

To estimate the difference in expected lifetimes for a cohort of patients treated with policy A versus policy B, one would simulate n patients with each policy, form the pair-wise differences, and construct the desired confidence interval on the average. CRN tries to reduce the variance of the differences by making the same patients under each policy use identical random numbers (RNs) for the same reasons. For example, if patient 1 has a probability of .03 of dying from HIV under both policies in month 7, then CRN prevents policy A from generating a RN of .02 (patient dies) and policy B from generating .47 (patient lives). By making them use the same RN, CRN focuses any difference in outcomes on true policy differences rather than differences in luck. We implemented full and partial CRN in a simulation of 10,000 HIV patients undergoing highly active antiretroviral therapy (HAART) and evaluated the mean survival time for patients starting HAART at any CD4 count vs. starting at 200. Full CRN ensured that every RN that may be used by a patient is identical between policies. For partial CRN, we just ensured that the probabilities of dying were identical.

Results:

The half-length of the 95% confidence intervals for the difference in mean survival for no CRN, partial CRN, and full CRN were .35, .18, and .12 years, respectively (the average difference in survival was .52 years). This implies, that it would take about 9 times as many replications using independent runs to achieve the same precision obtained using full CRN and about 4 times as many compared to partial CRN.

Conclusion:

Though in general the use of CRN is not guaranteed to reduce the estimated variance, we have empirical support for its success when applied to individual cohort simulations such as the one presented here.


See more of Poster Session - Clinical Strategies; Judgment and Decison Making
See more of The 26th Annual Meeting of the Society for Medical Decision Making (October 17-20, 2004)