Candidate for the Lee B. Lusted Student Prize Competition
Purpose: Develop an instrument using Multiple Signal Detection (MSD) measuring how people assess genetic breast cancer risk.
Methods: Widespread innumeracy creates problems when people estimate risk as probabilities. This technique asks people to make simple, ordinal "gist" assessments of low (L), medium (M), or high (H) risk. The model provides quantitative measures of sensitivity and biases in judgment criteria. The MSD model quantifies the ability to discriminate between risk categories (d') and biases in the criteria used to make such discriminations. In Study 1, untrained participants rated scenarios. In Study 2, before rating scenarios, participants were randomly assigned to either Intelligent Tutoring System (ITS); National Cancer Institute (NCI) website; or control groups. Our MSD model provides three discriminability measurers: d'(L-M) is sensitivity in distinguishing between low and medium risk scenarios; d'(M-H) and d'(L-H) assess sensitivity in distinguishing between low and medium risk, and low and high risk scenarios respectively. We developed 12 scenarios about breast cancer and genetic risk, describing women of low, medium, and high risk. Each scenario was equated in word length and reading level and included a woman's name, age, ethnicity, hometown, and personal and family health facts. The four low, medium, and high risk scenarios had Pedigree Assessment Tool scores of 0, 3–5, and 8–10 respectively; and mean Gail Model lifetime risk estimates of 4.9%, 7.4%, and 12.9% respectively.
Results: In Study 1, d'(L-M)=0.68 and d'(M-H)=0.53, both indicate a modest ability to distinguish between these risk levels. d'(L-H)=1.21 indicates a better ability to distinguish between low and high risk scenarios. Study 2 control group results were d'(L-M)=0.56; d'(M-H)=0.51; and d'(L-H)=1.07. Nonparametric permutation tests revealed significantly increased discriminability in all risk categories for ITS compared to the control group, d'(L-M)=1.09; d'(M-H)=0.97; and d'(L-H)=2.06, p<0.0005. The figure shows distributions for the ITS group with d'(L) set at 0 as a reference point.
Conclusions: Untrained participants modestly distinguished among low, medium, and high breast cancer risk levels. The ITS and NCI interventions significantly improved discriminability, while leaving the criteria relatively unaffected. Using MSD with these scenarios is a powerful tool for assessing the effectiveness of interventions because it provides separate measures of discriminability and criteria bias at different risk levels that are theoretically compatible with ordinal gist-based discrimination judgments. 
See more of: The 34th Annual Meeting of the Society for Medical Decision Making