RESOLVING STRUCTURAL BIAS DUE TO COMMON RANDOM NUMBERS WHEN SIMULATING EVENTS WITH NON-ORDERED OUTCOMES

Purpose:    To address structural bias that occurs when simulating events with multiple, mutually exclusive, non-ordered outcomes using common random numbers.

Methods:   The use of common random numbers (CRN) is a powerful variance reduction technique for simulation, and the resulting synchronization of events across counterfactual model runs allows for causal inference.  However, the structure of competing events can result in bias when CRN is used.  In particular, the simulation of events with multiple, mutually exclusive, non-ordered outcomes is prone to bias under CRN.  Using examples from the Maternal Health Policy microsimulation model we explore the construction of various event simulations and present solutions for removing structural bias.

Result:   Constructing a typical cumulative probability distribution (CPD) allows mutually exclusive events to be simulated easily.  Given an inherent ordering of outcomes (e.g. severity levels of a disease), constructing the CPD in decreasing order of severity results in unbiased simulations under CRN across counterfactual scenarios at both the aggregate and individual level.  However, for outcomes with no inherent ordering (e.g. type of obstetric complication, choice of contraceptive method, etc.) the construction of a typical CPD will result in biased outcomes at the individual level.  CRN ensures that for each event, the same point in the CPD is sampled in every simulation.  Therefore, only changes in the probability of an event in the CPD (such as the reduction of one type of obstetric complication) will result in an alternate outcome.  However, given a fixed random number and static order of events in the CPD, any alternate outcome is likely to be adjacent or near to the original outcome.  In other words, the chance of each alternate outcome occurring is no longer proportional to its probability, but rather is highly influenced by its arbitrary position in the CPD relative to the original outcome.   To correct for this structural bias we developed a method to divide and randomly allocate outcome shares within a CPD, and demonstrate how this method resolves bias at the individual event level.

Conclusion:   The use of CRN is an important modeling technique, but care should be taken to avoid unintended consequences and ensure that structural bias does not occur.  The random allocation of shares of non-ordered outcomes within a CPD is a feasible approach to address this problem.