Purpose: For cost-effectiveness analyses (CEA) that use observational data the key methodological challenge is to minimize selection bias. Propensity score (Pscore) methods can reduce selection bias due to observable differences between treatment groups; but the true Pscore model is generally unknown. Doubly robust (DR) methods exploit information in the Pscore and the response models, and provide unbiased estimates if either model is correctly specified. These methods hold promise for CEA, where selection bias needs to be minimized for the cost as well as the effectiveness endpoint. DR methods have not been examined before in this context.

Method: One implementation of DR methods is inverse probability of treatment weighting (AIPTW). The simple IPTW estimator weights observed cost and effectiveness endpoints with the inverse of the Pscore, to estimate incremental costs and effectiveness. AIPTW extends this by adjusting the formula with weighted predictions from the regression models of the respective endpoints. If a response model is correctly specified, adding this term can reduce bias. The adjustment also stabilises extreme Pscore weights, which can improve the precision of the IPTW estimator.

   To compare the methods in a CEA, we evaluate Drotrecogin alfa activated (DrotAA), a pharmaceutical intervention for critically ill patients with severe sepsis. We use data from a published observational study (n=1,898). Potential confounders were selected a priori (e.g. age, APACHE II severity score). Higher order terms and interaction terms were considered, and regression models for both cost and effectiveness were selected by cross-validation. A two-part model was chosen for the QALY and a generalized linear model with gamma distribution for the costs. To maintain correlation between costs and effects, confidence intervals (CI) were constructed by nonparametric bootstrapping.

Result: The incremental net benefit (INB) (λ=£20,000 per QALY) for DrotAA following IPTW was -£4796 (95% CI: - 23927 to 14969). After applying AIPTW, the estimated INB was £4936. Stabilizing the extreme Pscore weights led to tighter CI (-3867 to 12229).

Conclusion: DR methods avoid relying solely on a correctly specified Pscore or response model, and can lead to different point estimates and narrower CI than IPTW. Recent work shows that DR methods, eg. collaborative targeted maximum likelihood, can minimize bias and be efficient even if neither the Pscore or response models are correct, offering further flexibility in CEA.

Purpose: In cost-effectiveness analyses (CEAs) involving continuous decision variables (such as screening rates or treatment thresholds), the strategies being evaluated are generally pre-specified using arbitrary thresholds or round numbers.  The objective of this study was to evaluate the potential gains in welfare, defined by average net monetary benefit (NMB), from direct-search optimization of continuous decision variables (cardiovascular disease [CVD] screening/treatment thresholds) compared to solely focusing on pre-specified strategies.

Method: We used a CVD micro-simulation model to estimate the lifetime health benefits (quality-adjusted life years [QALYs]) and screening, treatment, and event costs under various multi-staged screening/treatment strategies for a representative cohort of 10,000 adults (aged 25-74 years) in the U.S. without history of CVD.  Screening/treatment strategies were defined by the numbers of individuals receiving non-laboratory-based or cholesterol-based risk assessment, and by the proportions of individuals ultimately receiving lipid-lowering and/or blood pressure treatment.  In total, 36 age- and sex-specific continuous decision variables collectively defined any screening/treatment strategy.  Fifty pre-specified strategies were determined based on commonly-used treatment thresholds and/or plausible screening/treatment cutoffs that spanned a considerable range of the decision variable space.   These strategies were compared to an optimized set of decision variables that was determined using the Nelder-Mead algorithm, a direct-search method that aimed to maximize average NMB (discounted at 3%, using a willingness-to-pay [WTP] value of $100,000/QALY).   Common random numbers were employed to produce stable results across model runs.

Result: Among the pre-specified strategies, the optimal option under conventional incremental CEA rules yielded discounted per-person averages of 20.422 QALYs, costs of $12,734, and average NMB of $2.0295 million.  The corresponding results from the direct-search optimization were 20.419 QALYs, costs of $11,456, and average NMB of $2.0305 million.  Extrapolated to the relevant U.S. population eligible for primary CVD prevention (~136 million adults), the total difference in average NMB between these approaches would be >$130 billion.

Conclusion: We found that direct-search optimization of multistage CVD screening/treatment thresholds resulted in meaningful gains in welfare (average NMB) compared to a traditional CEA of pre-specified strategies.  Future CEA studies involving many (>10) continuous decision variables might also benefit from employing direct-search or other optimization algorithms, although the gains in NMB should be weighed against potential losses from increased complexity of model results and subsequent clinical guidance (i.e., nuanced screening/treatment guidelines).

Purpose: To evaluate the role of aspirin together with the combined management of hyperlipidemia and hypertension in patients with type 2 diabetes.

Method: We present a Markov decision process model to determine the optimal start times for the combination of aspirin and the most common cholesterol and blood pressure medications for patients with type 2 diabetes. Health states were defined by cholesterol, blood pressure, A1c, and other risk factors used by the United Kingdom Prospective Diabetes Study risk model. Transition probabilities and treatment effects were estimated from a longitudinal clinical dataset from the Mayo Clinic electronic medical record. Cost parameters and disutilities were taken from secondary sources. The objective of the model was to maximize expected rewards over the course of the patient’s lifetime. Rewards were defined by the difference in benefits of increased quality-adjusted life years (QALYs) to first event (including stroke, CHD, gastrointestinal bleed, and death from all causes) based on a societal willingness-to-pay factor, minus costs of medication.  One-way sensitivity analysis was performed for the risk reduction factors for stroke and CHD, and the probability of gastrointestinal bleed.

Result: We computed the optimal treatment guidelines assuming availability of aspirin, statins, fibrates, ACE Inhibitors, Thiazides, and Beta-Blockers. For the base case the average incremental effect of adding aspirin is an increase of 0.736 QALYS and a decrease of $291 for males, and an increase of 0.434 QALYs and a decrease of $675 for females. Depending on individual CHD and stroke risk, females should initiate aspirin between the ages of 40 and 48; males should initiate aspirin at age 40, regardless of risk. Relative to the baseline, varying risk reduction for stroke from 0.85 to 1.06 resulted in a change in QALYs from 0.212 to -0.228. Varying risk reduction for CHD from 0.75 to 0.90 resulted in a change in QALYs from 0.215 to -0.230. Varying annual probability of gastrointestinal bleed from 0.0002 to 0.0005 resulted in a change in QALYs from 0.057 to -0.104. Across all cases the latest start times for males and females are 45 and 54 respectively.

Conclusion: Aspirin is beneficial for all patients with type 2 diabetes.  The optimal time for initiation depends on the patient’s individual risk level and assumptions about aspirin effectiveness and risk of gastrointestinal bleeding.

Purpose:   Clostridium difficile infection (CDI) affects 500,000 Americans every year, and is responsible for nearly 20,000 deaths annually. Although there are guidelines to control CDI outbreaks in a hospital, there is a strong need to develop rigorous methods to assess the efficacy of these strategies. The purpose of this study is to evaluate the performance of strategies to mitigate disease spread in a hospital. 

Method:   We propose an agent-based simulation to model the effects of infection control strategies to minimize disease transmission rates, CDI-related mortality, and exposure. Agent-based simulation is ideal for studying the interaction between patients that results in disease transmission, because it tracks the behavior of patients, health-care staff, and visitors in the hospital. Patients arrive to the hospital, stay for a random duration, and then leave the system. During their stay, patients may develop CDI or contract CDI from other infected or exposed individuals in the hospital. We analyze the efficacy of various infection control strategies including prophylactic vancomycin treatment, patient isolation, routine bleach disinfection of rooms, and increased hand hygiene measures, and how these strategies affect outcomes such as infection rates and length of stay (LOS). We use data from admissions records from the Wisconsin Hospital Association, which include data from hospitals in the state of Wisconsin from January 2007 to June 2010, covering over two million hospital admissions. 

Result:   Comparing individual strategies to the base case (no strategy), our preliminary results are as follows: Vancomycin treatment leads to a 12.9% reduction in average LOS over all patients, 8.9% less CDI cases, and 5.5% fewer relapse CDI. Infected patient isolation leads to a 14.3% reduction in LOS, 4% fewer CDI cases, and 29.1% fewer relapse CDI. Routine bleach disinfection leads to a 16.6% reduction in LOS, 6.3% fewer CDI cases, and 31% fewer relapse CDI.  Increased hand hygiene leads to a 6.1% reduction in LOS, 5% fewer CDI cases, and 10.9% fewer relapse CDI.  Finally, a comprehensive strategy leads to 59.7% reduction in average LOS, a 25.2% reduction in new CDI, and a 74.1% reduction in relapse CDI.

Conclusion:  Our agent-based model provides a rigorous analytical method for evaluating the efficacy of a customized strategy for combating CDI outbreaks in a hospital, thus leading to shorter LOS, fewer infections, and fewer relapses.

Purpose: To develop a maximum expected utility (MEU) model for assessing the value of diagnostic tests, and use this model to evaluate screening versus diagnostic mammography.

Method: We collected the records of 2,378 consecutive patients who underwent screening and follow-up diagnostic mammographic examinations from 2005-2008, which contained demographic risk factors and mammographic findings. Based on these features, we used a Bayesian network (BN) to estimate the risk of malignancy, constructed a receiver operating characteristic (ROC) curve using the BN estimated probabilities, and determined the optimal operating point at which expected utility was maximized. We first trained and tested two BNs (one screening and one diagnostic) using the tree augmented naïve Bayes (TAN) algorithm and 10-fold cross-validation. We generated ROC curves and calculated area under each ROC curve (AUC). Then, we assigned utility values for each category of findings (True Negative (TN), False Positive (FP), False Negative (FN) and True Positive (TP)) as follows. TN findings were chosen as our baseline and assigned a utility of zero. Based on the literature, the utility of FP was assigned a loss of ten days due to physical discomfort and anxiety. We used the previously developed and validated University of Wisconsin Breast Cancer Simulation (UWBCS) model to estimate the utility of FN as a loss of 2.52 years. We assumed the utility of TP was U(FN) × (1-α), 0≤α≤1, where α is an unknown parameter representing the overall effectiveness of breast cancer treatment. Finally, we found MEU at the optimal operating point on the ROC curve that intersected the line with slope [(U(TN)-U(FP))/(U(TP)-U(FN))] x [(1-p)/p], where p is prevalence of breast cancer.

Result: Diagnostic mammography was overall more accurate than screening mammography (AUC: 0.936 vs. 0.773, p<0.001). The MEU of both diagnostic and screening mammography increased as α increased. MEU of diagnostic mammography exceeded that of screening mammography for all values of α, with the difference approximately equal to 0.012 when α≥0.5.

Conclusion: Diagnostic mammography has higher accuracy and MEU when compared to screening mammography. Our analysis indicates that MEU methods can provide a framework to assess the value of diagnostic tests in other clinical areas, making use of the relative consequences of correct and incorrect diagnosis.