Purpose: To evaluate whether the advance directive documents (AD) generated by an interactive, online decision aid for advance care planning, Making Your Wishes Known (MYWK), reliably reflects an individual's values/preferences.
Method: English speakers ≥age 30 completed MYWK at three visits, two weeks apart. Consistency across study visits was calculated for the General Wishes component of the AD by comparing responses for each visit. Generalizability Theory (G-theory) was used to examine the reliability (or dependability) of the Specific Wishes for the treatment component of the AD. G-theory examines multiple sources of variation comparatively in order to gain a more detailed understanding of the dependability of the scores than is possible using test-retest methods alone. Dependability indices were calculated, specific to the study design where variation is attributed to differences in the individuals completing the MYWK program, five clinical scenarios, and three time points. A generalizability equation was constructed to predict the reliability for specific time points and scenarios, as well as to provide a quantifiable assessment of the stability of the measure across time.
Result: 33 participants completed the study. General Wishes remained stable with 94% selecting the identical response at each of the three visits. For Specific Wishes, a pairwise comparison of correlation coefficients across time points within scenario showed no significant difference between comparisons for any of the five scenarios. Using G-theory, the reliability of a person’s Specific Wishes is suggested to be high (79%), but not ideal (> 90%) when assessed for a given scenario on one occasion. Temporal stability is lower (61%) suggesting that time effects play an important role in determining the Specific Wish score so that a patient’s score is subject to significant variability. Overall, however, the major source of variation in Specific Wishes scores is due to scenario effects.
Conclusion: Advance directives generated by an online decision aid demonstrate good reliability, with the highest stability for General Wishes statements. There was moderate dependability for Specific Wishes regarding medical treatments within the specific clinical scenarios, but lower dependability for Specific Wishes across multiple occasions and clinical scenarios. Further research will explore whether observed variability reflects limitations of the decision aid, or a more fundamental inconsistency in how individuals make prospective choices for specific treatment options.
See more of: The 34th Annual Meeting of the Society for Medical Decision Making