Purpose: To estimate the efficacy of electronic aids to smoking cessation, as an input into a cost-effectiveness analysis of such interventions.

Method: A prior systematic review identified studies evaluating electronic aids to smoking cessation, (websites, computer-based tailored advice, chat rooms, email / SMS communications, etc). A classification system was developed for these aids with five levels, ascending from single-component interventions providing generic advice (e.g. static website) to interventions with multiple components providing tailored feedback through several channels (e.g. interactive website + email + chat room). To synthesise this evidence and estimate class-level treatment effects, a Bayesian mixed-treatment comparison was constructed. This involved fitting a proportional-hazard Weibull survival model on sustained abstinence, as this was the main outcome of interest for the cost-effectiveness model. For some treatment classes, evidence on sustained abstinence was lacking, but point abstinence rates were available. A log-odds treatment effect was fitted to the latter type of outcome, along with a correlation structure between treatment effects on the two outcome types. This allowed treatment effects on sustained abstinence to be estimated for all intervention classes.

Result: 51 studies were included in the analysis, with 127 arms. 62 arms reported sustained abstinence, of which 51 arms also reported point abstinence. The mean shape for the Weibull survival model was 0.18 (95% credible interval (CrI) 0.09-0.32), which was consistent with the hypothesis that quitting is hardest initially and becomes easier to sustain with time.  There was an inverse relationship between the mean hazard ratio and the class of treatment, with estimates ranging from 1.15 (95% CrI 0.87-1.45) for the class one hazard ratio to 0.81 (95% CrI 0.68-0.93) for the class five hazard ratio.

Conclusion: Bayesian methods allow for uncertainty in treatment effects to be quantified whilst incorporating treatment classes, multiple outcomes, and repeated measurements. Application to data on electronic smoking cessation aids demonstrated that such interventions are likely to improve sustained abstinence, and that increased intensity may lead to better outcomes. Further studies are required to determine the incremental benefits of more intensive electronic interventions; the design of these trials should be informed by value of information analyses.

Purpose:    Computer-based tools to assess venous thromboembolism (VTE) risk have been shown to increase VTE pharmacoprophylaxis rates and decrease VTE incidence in hospitalized patients.  However, these tools are dependent on the quality of electronically available clinical data; if the clinical data needed for accurate VTE risk assessment is incomplete, VTE risk may be underestimated in at-risk patients.  We hypothesized that a physician-enhanced clinical decision support (CDS) tool would identify more patients as moderate and high-risk than computer-based tools alone that do not incorporate physician input.

Method:    Our institution implemented a physician-enhanced CDS tool that required physicians to stratify hospitalized patients as high, moderate, or low-risk for VTE based on assessment of VTE risk factors enhanced with individual physician judgment.  We compared rates of VTE pharmacoprophylaxis and VTE incidence in adult patients hospitalized 4 months prior and 3 months after implementation of the physician-enhanced CDS tool.  The study sample was restricted to adult hospitalized patients who were determined to be at low-risk for VTE, and would not have received pharmacoprophylaxis based on a computer-based tool alone.  

Result:    During the 7-month study period, 25.4% (n=2423) of hospitalized adult patients would have been deemed low-risk for VTE based on a computer-based risk assessment tool alone.  After implementation of the physician-enhanced CDS tool, physicians stratified 324 (27.1%) of these patients as moderate-risk and 99 (8.3%) as high-risk for VTE.  After implementation of the physician-enhanced CDS tool, the rate of VTE pharmacoprophylaxis in this sample increased (27.1% to 35.1%, p<0.01) and the incidence of in hospital VTEs decreased (0.98% to 0.25%, p=0.02).

Conclusion:    Patients identified as low- risk for VTE solely by computer-based algorithms may miss patients that physicians determine to be at risk for VTE and warrant VTE prophylaxis.  Physician-enhanced clinical decision support tools that involve clinician-guided risk assessment may outperform computer-based VTE risk stratification algorithms.  However, adverse events that might occur as a result of expanded VTE pharmacoprophylaxis from CDS tools need to be determined.

Purpose: To compare knowledge of the risks/benefits of tamoxifen when presented as the sole alternative to chemoprevention versus as one of two alternatives.   

Method: We conducted two studies testing a decision aid (DA) designed to inform women about breast cancer chemoprevention. Participants in both studies were at high risk for breast cancer. Study 1 occurred prior to the STAR trial results and thus only described the risks/benefits of tamoxifen. Following the STAR trial, which showed that raloxifene was equivalent to tamoxifen in decreasing the risk of invasive breast cancer (with a similar side effect profile), we conducted Study 2 using a second DA that described the risks/benefits of both tamoxifen and raloxifene. Study 2 also tested two approaches to reducing cognitive burden: 1) presenting risk/benefit data for tamoxifen and raloxifene in separate pictographs vs. collapsing them into one pictograph and 2) including a summary table of information. After reviewing the DA, participants in both studies answered the following question: “Who is more likely to experience (health condition: e.g. endometrial cancer): A person who took tamoxifen, a person who did not take tamoxifen, equally likely.”   

Result: 623 women participated in Study 1 and 690 participated in Study 2. As shown below, women’s knowledge of the risks/benefits of tamoxifen was reduced when they learned about both tamoxifen and raloxifene. Percent answering each question correctly  

	Only Tamoxifen  (S1)	Tamoxifen + Raloxifene (S2)
Endometrial cancer	72.8	45.5
Blood clotting	70.8	71.6
Cataracts	68.4	68.6
Hormonal symptoms	70.0	67.3
Bone fractures	69.4	42.8
Breast cancer	81.0	73.2

   In Study 2, knowledge did not differ based on whether participants received the information collapsed across drug or separately for each drug or whether participants received a summary table.   

Conclusion: The STAR trial showed that women at high risk of developing breast cancer now have a second drug they can use to reduce this risk.  Unfortunately, inclusion of information about this second alternative reduces people’s ability to process all of the information relevant to their decision.

Purpose: To evaluate a preference-tailored decision tool for improving the quality of surgical treatment decisions for patients with early stage breast cancer.

Method: A web-based decision tool was developed over a one-year period with input from health communication experts, breast cancer clinicians, and women with breast cancer . The tool included an interactive conjoint-analysis based preference elicitation exercise that provided users with feedback about their preferences for treatment attributes in real time.  Newly diagnosed early stage breast cancer patients of a comprehensive cancer center were recruited and randomized to view the tool before or after completing a survey. Mean scores for key outcome measures, including surgical treatment knowledge (4 true/false questions), decision satisfaction (12 questions each with a 5-point Likert scale from strongly agree to strongly disagree), and preference-concordant decisions, were compared between the groups using t-tests. Concordance between preferences and surgical choices was evaluated using the chi-square test.

Result: To date, 70 subjects have been recruited with complete information available for 58. Their mean age was 57 years, 60% had a college degree or more, and 86% were white. Those viewing the website first had higher scores on several decision outcomes than those taking the survey first (Table). Knowledge scores were also higher among those viewing the website before the survey (3.0 vs. 2.61, p=.23). The risk of recurrence was the most important treatment attribute, followed by retaining the natural breast, in both groups. Concordance between treatment choice and conjoint-analysis generated treatment was 65% for website first and 61% for survey first groups.

	Website before survey	Survey before website
	Response 1-5 (strongly agree – strongly disagree)
I’m unsure what decision to make	4.12	3.35*
The surgical treatment decision is hard for me	3.76	2.80
I’m aware of the choices I have to treat my breast cancer	1.61	1.83*
I am satisfied with my surgical treatment decision	1.44	1.65
I feel the surgical treatment decision matches my values	1.32	1.77^
*P<0.05, ^P<0.10

Conclusion: A tool designed to improve the quality of surgical breast cancer treatment decisions by focusing on improving knowledge and preference-concordant decisions appears effective in this pilot study. Further work to assess the impact of the tool in larger and more diverse populations is needed.

Purpose: We sought to compare, in a randomized trial, two techniques for eliciting and clarifying patient values for decision making about colorectal cancer (CRC) screening: conjoint analysis and a rating and ranking exercise.

Method: Based on our past research and a review of the literature, we identified 6 key attributes of CRC screening tests: 1) ability to reduce CRC incidence and mortality; 2) test-related discomfort; 3) nature of the test (where performed, time required); 4) test frequency; 5) major complications; and 6) out of pocket costs. Using our decision lab registry and university email lists, we recruited adults ages 48-75 who were at average risk for colorectal cancer for a written, mailed survey.  Eligible participants were given basic information about CRC screening and then randomized to complete either a choice-based conjoint analysis with 16 discrete choice tasks or a rating and ranking exercise. Outcomes included most important attribute, as determined from conjoint analysis or participant ranking; values clarity (sub-scale of the decision conflict scale), intent to be screened, and unlabelled test preference, as assessed on a post-task questionnaire. Conjoint analysis most important attribute was based on individual patient-level utilities generated using multinomial logistic regression and a hierarchical Bayesian modeling approach implemented in Sawtooth software.

Result: 114 respondents were eligible and randomized (54 to conjoint analysis and 60 to rating and ranking) and 104 (50 conjoint analysis, 54 rating/ranking) completed and returned questionnaires. Mean age was 57 (range 48-73), 70% were female,  88% were White, and 71% were college graduates. 62% were up to date with CRC screening, with most having had colonoscopy. Ability to reduce CRC incidence and mortality was the most frequent most important attribute for both the conjoint analysis (56% of respondents) and rating/ranking (76% of respondents) groups, but this proportion differed significantly between groups (absolute difference 20%, 95% CI 3%, 37%, p =0.03). There were no significant differences between groups in proportion with clear values (p=0.352), intent to be screened (p=0.226) or unlabelled test preference (p=0.521)

Conclusion: A choice-based conjoint analysis task produced somewhat different patterns of attribute importance than rating and ranking exercises, but had little effect on other outcomes in this small trial. Larger trials comparing these values clarification techniques and measuring their effect on screening behavior are warranted.