TRA-2
TOP RATED ABSTRACTS II
Purpose: Because risk often varies across patients enrolled in randomized trials, average population trial and cost-effectiveness analysis (CEA) results do not apply to many patients. Risk prediction models make it possible to incorporate individual risk and clinical effectiveness information to compute individualized CEA and identify patients for whom therapy is most appropriate. The expected value of individualized care (EVIC) refers to the incremental monetized value of customizing care in this manner, compared to a uniform recommendation (treat all or none) based on the population average incremental cost-effectiveness ratio (ICER). We explore factors influencing EVIC.
Methods: We developed a general framework to calculate individualized ICERs as a function of individual outcome risk. For a case study (tPA vs. streptokinase to treat possible myocardial infarction), we used simulation to explore how EVIC is influenced by risk model discriminatory power (c-statistic), population outcome prevalence, model calibration and willingness-to-pay (WTP) thresholds. We characterized individual risk using beta distributions, the parameters for which we estimated from population prevalence and the risk model c-statistic. We derived this relationship empirically from a database of 25 large randomized clinical trials. We assumed that treatment effect (relative risk reduction) and life expectancy do not vary according to patient risk.
Results: For unbiased models (figure), EVIC is always non-negative and improvements in discrimination better targets treatment, increasing EVIC. In our example, lower population outcome prevalence also increases EVIC. EVIC also depends on the WTP threshold; when the WTP and population average ICER are close, EVIC increases because individualized information more often changes treatment decisions. In contrast to unbiased prediction models, miscalibrated models can introduce mistakes and hence have a negative EVIC. When the population average ICER is near the WTP threshold, decision making is more sensitive to bias and these “mistakes” are more common (making EVIC more negative). In simulated examples, EVIC decreased for model c-statistic values of 0.6-0.7 and increased for c-statistic values of 0.8-0.9.
Conclusions: In general, higher predictive model c-statistic values produce higher EVIC values. This benefit is greatest when the population ICER is near the WTP threshold. When models are miscalibrated, greater discriminating power can paradoxically reduce the EVIC under some circumstances.
Figure: EVIC as a function of c-statistic at several levels of outcome prevalence and WTP thresholds
Method: We developed a 5-compartment dynamic transmission model of Ebola for the 2014 Liberian outbreak and performed literature review to characterize model inputs and their uncertainty. We matched 2-week moving averages of new Ebola cases reported by the World Health Organization both before widespread burial and social distancing interventions began in Liberia (prior to September 2014) as well as afterwards (through mid-May 2015) by performing 10,000 Neldor-Mead search calibrations from random starting sets of inputs. By simultaneously calibrating to both periods, we recovered natural history parameters and intervention effectiveness. The objective function of the calibration was a weighted sum-of-squares where weights were the inverse of the standard error of the observed estimates under the binomial distribution. For analyses of alternative timings of interventions, we sampled 1,000 calibrated parameter sets with replacement from the 10,000, weighting the sampling by an approximation of the likelihood function so that better-fitting sets were more likely to be sampled.
Result: Compared to the observed 10,604 cumulative Ebola cases from the current outbreak in Liberia, our model predicts 10,519 cases [95%CrI: 9,755-10,992]. If interventions had been implemented earlier by 1 month, total cases are predicted at 1,904 [95%CrI: 1,359-2,951]. At 2 months earlier, these figures are 485 cases [95%CrI: 273-1,041].
Conclusion: Initiating safe burial and social distancing interventions earlier via better surveillance and epidemic preparedness has the potential to substantially decrease the impact of future Ebola epidemics in urban African settings.
Method: Machine learning methods were used to develop the model to forecast occurrence of major depression, measured by Patient Health Questionnaire 9-item score≥10. Predictors were selected using a correlation-based subset evaluation method from 20 risk factors of depression. Two linear models, Ridge logistic regression and multilayer perceptron, and two nonlinear models, support vector machine and random forest, were trained and validated on data pooled from two safety-net clinical trials of diabetes and depression (N=1793). The model with the best overall predictive ability, measured by area under receiver-operating curve (AUROC), was chosen as the ultimate model. Depression identification rate and measures relevant to provider resource and time were compared between a model-based policy that screens only patients predicted as being depression and alternative policies. These policies include universal screening and partial screening based on certain risk factors of depression such as depression history, diabetes severity, or either criteria.
Result: Seven predictors were selected to develop the prediction model: 1) gender, 2) Tolbert diabetes self-care 3) number of diabetes complications, 4) previous diagnosis of major depression, 5) number of ICD-9 diagnoses in past 6 months, 6) chronic pain, and 7) self-rated health status. Ridge logistic regression with the above seven predictors had the best overall predictive ability (AUROC=0.81) and was chosen as the ultimate model. Compared to universal screening, the model-based policy can save about 50-60% of provider resources and time but will miss identification of about 30% of depression cases. Partial-screening policy based on depression history alone yielded a very low rate of depression identification. Two other partial screening policies have depression identification rates similar to model-based policy but cost more in resources and time.
Conclusion: The depression prediction model developed in this study has compelling predictive ability. By adopting the model-based depression screening policy, healthcare providers can better prioritize the use of their resources and time while increasing efficiency in managing their patient population with depression.
Method: We used PA Medicaid claims data from 2007–2012 to identify individuals diagnosed with HCV, individuals who received HCV therapies, and those who developed advanced liver disease due to HCV infection. To estimate the current HCV prevalence, we used an innovative approach of combining the results of claims data with a validated microsimulation model that accurately predicted national HCV prevalence in the United States. In this process, we accounted for trends in Medicaid enrollment, and adjusted the rates of treatment contraindications, such as substance abuse, that are often higher in Medicaid populations. We calibrated our model such that it simulated the observed number of patients diagnosed with HCV, and “hard outcomes” (liver transplants, hepatocellular carcinoma, decompensated cirrhosis) from 2007–2012 claims data. Our model included historic as well as current HCV screening and treatment recommendations. From the calibrated PA model, we estimated the number of patients who will need treatment in 2015 and beyond by disease stage (represented by fibrosis scores F0–F4) and by HCV genotype.
Result: Our calibrated model matched the number of individuals with HCV diagnoses based on PA Medicaid claims data at 26,400 in 2012. The model estimated 22 liver transplants in 2012, closely matching the true incidence found in claims data. Our model estimated that 46,400 beneficiaries were infected with HCV in 2015, of whom 65% were aware of their disease, and 72% were treatment naïve. In the following 10 years, 8,500 new patients would be added to PA Medicaid either because of HCV screening or new enrollments.
Conclusion: We provide a novel approach to estimate the prevalence of HCV by using a combination of claims data and simulation modeling. Our results can assist state Medicaid programs in effective allocation of their resources to manage HCV patients in a rapidly changing clinical and policy environment.
Methods:
Caregivers of a
4-14 year-old child diagnosed with ADHD were recruited from pediatric primary and
mental health clinics and family support groups across Maryland. A case 1, balanced
incomplete block design (BIBD), best-worst scaling (BWS) experiment assessed caregivers'
most important concerns when initiating ADHD treatment. Participants completed
16 choice tasks, each showing 6 of the 16 concerns from which one most and one
least important concern was selected. Demographic characteristics, caregiver-reported
improvements resulting from medication, and additional desired ADHD changes also
were reported. Preference utilities were estimated using conditional logit,
effects coding, and assuming sequential best-worst responses. Scores were ranked to assess
relative importance. Latent class analysis (LCA) was conducted to determine if
there were distinct segments that prioritized concerns differently. Reported
outcomes were estimated based on the observed impact of treatment and
additional desired changes.
