ASSESSING THE QUALITY OF DECISION SUPPORT TECHNOLOGIES USING THE INTERNATIONAL PATIENT DECISION AID STANDARDS INSTRUMENT (IPDASI)

Purpose To describe the development, validation and inter-rater reliability of an instrument to measure the quality of decision support technologies (decision aids). Scale development study, involving construct, item and scale development, validation and reliability testing. There has been increasing use of decisions support technologies – adjuncts to the discussions clinicians have with patients about difficult decisions. A global interest in developing these interventions exists, by both for-profit and not-for-profit organisations, creates the potential for conflicts of interest. It is therefore essential to have internationally accepted standards to assess their quality and potential bias. Method Twenty five researcher-members of the International Patient Decision Aid Standards Collaboration worked together to develop the instrument (IPDASi). A four stage validation study was conducted. In the fourth stage (reliability study), eight raters assessed thirty randomly selected decision support technologies. Results IPDASi measures quality in 10 domains, using 47 items, and provided an overall quality score (scaled from 0 to 100) for each intervention. Overall IPDASi scores ranged from 33 to 82 across the decision support technologies sampled, enabling discrimination. The inter-rater intraclass correlation for the overall quality score was 0.80. Correlations of domain scores with the overall score were all positive (0.31 to 0.68). Cronbach’s alpha values for the 8 raters ranged from 0.72 to 0.93. Cronbach’s alphas based on the means in the domains ranged from 0.50 to 0.81, indicating that the domains, although well correlated, measure different aspects of decision support technology quality. A short version (19 items) was also developed that had very similar mean scores to IPDASi and high correlation between short score and overall score 0.87 (0.7 to 0.92). Conclusions This work demonstrates that IPDASi has the ability to assess the quality of decision support technologies and could be considered for certification purposes. IPDASi could provide formative feedback about domains in which decision support technology developers could make improvements in subsequent versions. A short-form may also be developed to support the development of quality standards and enhance patient safety.