Concurrent Oral Presentation I. METHODS: DECISION MAKING AND DIAGNOSIS

Tuesday, October 20, 2009: 4:00 PM
Grand Ballroom, Salon 6 (Renaissance Hollywood Hotel)

* Candidate for the Lee B. Lusted Student Prize Competition

Session Chairs:
Amber E. Barnato, MD, MPH, MS and David O. Meltzer, MD, PhD
4:00 PM
* A GAME-THEORETIC FRAMEWORK FOR ESTIMATING THE WILLINGNESS-TO-PAY FOR HEALTH AND FOR EXPANSION
Reza Yaesoubi, MSc and Stephen D. Roberts, PhD, North Carolina State University, Raleigh, NC

Purpose: We developed a framework which answers two fundamental questions on willingness-to-pay (WTP): for a particular disease, how much is a health purchaser (e.g., a government, a health insurance system, or a private health insurer) willing to pay (1) for one unit of health (WTP for health)? and (2) for expanding the consumption level of a related medical intervention by one percent (WTP for expansion)? The WTP for health can be used to determine the medical guidelines and the price of newly-developed medical technologies such that the health purchasers find them worthwhile to be included in their offered health plans. The WTP for expansion can be employed by health providers in determining the optimal investment level in expanding a medical intervention through opening new facilities, advertisement, etc.

Method: We proposed a game-theoretic framework in which the health purchaser and the population enter into a two-move game. The health purchaser makes the first move by offering a contract (i.e., the set of medical interventions and the corresponding coinsurance rates) and then, having observed the contract and her health status, each individual decides which medical alternative to undergo. By mapping the equilibrium of this game to the observed health purchaser’s contract and the consumption level of each intervention, we estimate the WTP for health and the WTP for expansion.

Results: We applied the proposed framework to Colorectal Cancer (CRC) screening tests for the 2005 U.S. population considering Colonoscopy, Sigmoidoscopy, and Fecal Occult Blood Test (FOBT). For CRC screening tests, we estimate the WTP for health to be $9,950 per QALY, and the WTP for expanding Colonoscopy to be $45.40 per percent of increase in Colonoscopy consumption, per person. The framework also predicts that both FOBT and Sigmoidoscopy will soon leave the market, as also conjectured by previous studies.  

Conclusions: For CRC screening tests, the medical guidelines and new screening tests should demand less WTP for health than $9,950 per QALY to be implemented by the health insurance system, given the current consumption levels of the CRC screening tests in the U.S. Also, the insurance system is willing to pay up to $45.40 per person for any action (such as opening a new Colonoscopy suite, educating the population, etc.) that increases the Colonoscopy consumption by one percent.

4:15 PM
* A MATHEMATICAL MODEL TO OPTIMIZE BREAST CANCER SCREENING POLICY
Turgay Ayer, MS, University of Wisconsin, Madison, WI, Oguzhan Alagoz, PhD, University of Wisconsin-Madison, Madison, WI and Natasha K. Stout, Ph.D., Department of Population Medicine, Boston, MA

Purpose: Breast cancer is the most common non-skin cancer and the second leading cause of cancer-deaths in US women. Although mammography is cost-effective for breast cancer detection, questions remain about whom to screen and how frequently. The possible screening policies are innumerous and difficult to directly assess. The purpose of this study is to investigate the optimal personalized mammography-screening policy for a woman's lifetime using mathematical modeling.

Methods: We formulate a finite-horizon Partially Observable Markov Decision Process (POMDP) model that maximizes a woman's quality-adjusted life-years (QALYs). A POMDP is a generalization of a Markov decision process that allows sequential decision making when the information regarding the true state of the system is incomplete. Model inputs include state transition probabilities and rewards, estimated using the University of Wisconsin Breast Cancer Simulation. Our POMDP model incorporates unobservable disease progression, two methods of detection (self or screen), and mammography test characteristics. We solve this POMDP optimally to find the optimal personalized screening policy.

Results: A woman's optimal screening policy follows a threshold structure where the optimal decision is to screen if the current breast cancer risk is greater than a certain threshold risk, and wait for 6-months, otherwise. The threshold risk for screening increases with age (Figure 1). For example, the optimal policy for a 40-year old woman is to screen, if her current risk of in-situ or invasive breast cancer is above 0.4%; and wait for another six months, otherwise. For an 82-year old woman, the screening thresholds increase to 11.6% and 5.1% for in-situ and invasive cancer risks, respectively. Our results also suggest younger women should be more aggressively screened than older women. In terms of QALYs, a 40-year old woman with a 10% in-situ cancer risk would attain 40.2 QALYs following our optimal screening policy, a gain of 0.5 QALYs over the QALYs achievable from US screening guidelines. These gains increase with the woman's risk. For example, a 40-year old woman with a 30% in-situ cancer risk would gain over 1 QALY to attain 39.6 QALYs if she followed our optimal screening policy compared with US guidelines.

Conclusion: Unlike many prior studies, our POMDP model provides optimal screening policies for individual patients and has the potential to improve women's health.

Figure 1. Optimal mammography-screening policy.

4:30 PM
* DISEASE BURDEN AND LIFETIME DIRECT MEDICAL COSTS ASSOCIATED WITH CHRONIC HEPATITIS C INFECTION AMONG IMMIGRANTS IN CANADA
Wendong Chen, MD, PhD, George Tomlinson, PhD, Jenny Heathcote, MD and Murray D. Krahn, MD, MSc, University of Toronto, Toronto, ON, Canada

Purpose: To estimate the disease burden and lifetime direct medical costs caused by hepatitis C virus (HCV) infection among immigrants in Canada through a Markov cohort model.

Method: A Markov model with one-year cycle length and lifetime horizon was constructed according to HCV epidemiology, pattern of care, and the natural history of HCV among immigrants in Canada. Various resources were used to estimate the input parameters: 2006 Canada Census for the distribution of immigrants by age; WHO report for HCV prevalence by country; literature review for HCV genotype distribution by country; frequency of anti-HCV tests in 2006 in Ontario for HCV detection; medical records of immigrant patients (n=927) for fibrosis distribution, pattern of care, and the prognosis after cirrhosis; publications reporting sensitivity and specificity for HCV antibody test, HCV-induced fibrosis progress, and efficacy of anti-viral therapy; a cost study of 30000 HCV patients in British Columbia, Canada  for direct medical costs. Microsimulation with sampling all the distributions of the parameters in the model was conducted with the number of trials representing the immigrant population (n = 5411710) in Canada in 2006 to estimate the loss of life years, lifetime risk of HCV-related complications, and increased lifetime direct medical costs in 2006 Canadian dollars associated with HCV infection among immigrants.  

Result: The Markov model included eight branches by age group. Each branch contained 50 health states with 197 variables. The microsimulation of the Markov model estimated immigrants with chronic HCV infection had a shorter life expectancy (33.07 years vs. 38.27 years), higher lifetime risks for hepatocellular carcinoma (42.1%), hepatic decompensation (33.5%), and liver transplant (47.0%), and higher lifetime direct medical costs (584628 dollars vs. 182422 dollars) when compared to immigrants free of HCV infection. Based on estimated HCV prevalence (2.2%) and genotype distribution (70.4% for genotype 1, 4, 5, or and/or 6) among immigrants, the microsimulation of  the Markov model taking 2006 immigrant population as a cohort estimated 649405.2 life years lost, an increase of 46778.8 million Canadian dollars in 2006, 39257 decompensated patients, 49734 hepatocellular carcinoma patients, and 55383 liver transplants due to HCV infection.

Conclusion: Our Markov model projected a significant loss of life expectancy and increase of lifetime direct medical costs due to chronic HCV infection among immigrants in Canada.

4:45 PM
* MODEL AVERAGING IN THE PRESENCE OF STRUCTURAL UNCERTAINTY ABOUT TREATMENT EFFECTS: IMPACT ON TREATMENT DECISION AND EXPECTED VALUE OF INFORMATION
Malcolm J. Price, MSc, University of Bristol, Bristol, United Kingdom, A. E. Ades, PhD, Medical Research Council, Bristol, United Kingdom and Nicky J. Welton, PhD, Bristol University, Bristol, United Kingdom

Purpose: We assess the relative impact of different parameterisations of treatment effects on the resulting decision, on decision uncertainty and on the expected value of conducting further research in order to reduce decision uncertainty, and demonstrate the use of model averaging to incorporate structural uncertainty about the parameterisation.

Method: We use a Bayesian approach to model estimation, model selection, and model averaging in the context of Cost-Effectiveness and Expected Value of Perfect Information (EVPI) analyses for asthma treatments. We use aggregate level data from a connected network of four treatments compared in three pair-wise RCTs. We assess the relative impact of several different parameterisations of treatment effects on the resulting decision, on decision uncertainty, and on the EVPI. Structural uncertainty about which parameterisation to use is accounted for using model averaging and we develop a formula for calculating the EVPI in averaged models. Marginal posterior distributions are generated for each of the cost-effectiveness parameters using Markov Chain Monte Carlo simulation in WinBUGS, or Monte-Carlo simulation in Excel.

Result: The standard errors of incremental net benefit using structured models is reduced by up to 8 or 9-fold compared to the unstructured model, and the expected loss attaching to decision uncertainty by factors of several hundreds. Model averaging had little impact on the optimal decision but there was considerable impact on the EVPI.

Conclusion: Alternative structural assumptions can have an overwhelming effect on model uncertainty and Expected Value of Information. Structural uncertainty can be accounted for by model averaging, and EVPI can be calculated for averaged models.

5:00 PM
* A RETROSPECTIVE RECORD REVIEW STUDY ON THE CAUSES OF DIAGNOSTIC ADVERSE EVENTS
Laura Zwaan, MSc1, Martine de Bruyne, MD, PhD1, Cordula Wagner, PhD2, Abel Thijs, MD, PhD3 and Danielle R.M. Timmermans, PhD4, (1)VU University Medical Center, Amsterdam, Netherlands, (2)NIVEL and EMGO Institute for Health and Care Research, Utrecht/ Amsterdam, Netherlands, (3)Department of Internal Medicine, VU University Medical Center, Amsterdam, Netherlands, (4)EMGO Institute/ VU University Medical Center, Amsterdam, Netherlands

Purpose: Diagnostic adverse events (DAEs) are an important error type as they are often considered to be preventable and the consequences are judged to be severe. Therefore, reducing the number and severity of DAE is particularly important. The design of prevention strategies aimed at reducing DAEs can possibly be rationalised by examining how the causes of DAE differ from the causes of other adverse event (AE) types. Therefore, our study focuses on the causes of DAE in comparison to the causes of other AE types.

Method: A three stage retrospective patient record review study of 7926 patient records was conducted. The method used in this study was based on the well-known protocol developed by The Harvard Medical Practice Study in New York in 1984. To determine whether the AE occurred, trained physicians reviewed randomly selected patient records. Subsequently, the causes were assessed using the Eindhoven Classification Model. This classification model defines 4 main causes categories (human, organizational, technical and patient-related factors). The main categories are divided into 20 subcategories based on the model of unsafe acts (Reason 1990) and the SRK-model (Rasmussen 1976). We compared the causes of DAEs with the causes of other AE types.

Result: The main categories human causes and organizational factors occurred more often in DAE than in other AE types (human: 96.3% versus 50,5%, p <0,0001; organizational: 25.0% versus 12.7% p< 0.005). Patient-related factors were more often involved in the occurrence of other AE types (30.0% versus 44.9% p; <0.05). The influence of technical factors was marginal in all AE types. The analysis of the subcategories of the causes showed that especially subcategories involving (transfer of) knowledge occurred more often in DAEs; Knowledge-based (p< 0.00001), coordination (p< 0.0001) and transfer of knowledge (p< 0.01).

Conclusion: The study showed a substantial contribution of human causes to DAE. Besides the human causes, organisational causes, in particular transfer of knowledge, occurred relatively frequent in DAEs. Possible prevention strategies should focus on expanding physicians' general knowledge to reduce the occurrence of knowledge-based mistakes and training of the non-technical skills to improve transfer of knowledge and coordination. However, more research on the development, implementation and effectiveness of interventions is needed.

5:15 PM
* DISCRIMINATION AND CALIBRATION OF ARTIFICIAL NEURAL NETWORKS IN MAMMOGRAPHIC DIAGNOSIS
Turgay Ayer, MS1, Oguzhan Alagoz, PhD2, Jagpreet Chhatwal, PhD3, Jude W. Shavlik1, Charles E. Kahn, Jr., MD, MS4 and Elizabeth S. Burnside, MD, MPH, MS2, (1)University of Wisconsin, Madison, WI, (2)University of Wisconsin-Madison, Madison, WI, (3)Merck Research Laboratories, North Wales, PA, (4)Medical College of Wisconsin, Milwaukee, WI

Purpose: In this study, we develop an artificial neural network (ANN) to estimate the risk of breast cancer based on mammographic findings and demographic risk factors, and assess how well our ANN can (1) discriminate between benign and malignant mammographic findings, and (2) generate well-calibrated probabilities that estimate the risk of breast cancer for individual findings.

Method: Our dataset consisted of 62,219 prospectively collected consecutive mammography findings matched with Wisconsin State Cancer Reporting System. We built a three-layer ANN with excessive hidden nodes (1000) because large networks are shown to perform better when presented to unseen cases. We trained and tested our ANN using ten-fold cross validation and kept a validation set to prevent overfitting. We compared the performance of our ANN to that of interpreting radiologists. We used area under the receiver operating characteristic curve (AUC), sensitivity, and specificity to evaluate discriminative performance of our ANN and interpreting radiologists. We calculated the accuracy of risk prediction (i.e. calibration) of our ANN using the Hosmer–Lemeshow (H-L) goodness-of-fit test.

Result: Our ANN demonstrated an AUC = 0.965 ± 0.001, which was significantly higher (P < .001) than that of the radiologists, AUC = 0.939 ± 0.011. Our ANN also demonstrated significantly high calibration as shown by a small H-L statistic (12.46) and high P-value (P=0.13, df=8).

Conclusion: Our ANN can effectively discriminate malignant abnormalities from benign ones and produce well-calibrated risk estimates for individual abnormalities. Our findings suggest that ANNs may have the potential to help radiologists improve mammography interpretation.