Purpose: Online systems are being used increasingly in medicine to disseminate practice-related resources including clinical guidelines and treatment protocols. Despite their widespread use, the quality of these online resources has not been evaluated. This study addresses this gap in the area of medical oncology. Specifically, we evaluated web-based oncology resources developed across a number of health care settings internationally, using a psychometrically robust and highly utilized evaluation tool.
Methods: The Appraisal of Guidelines for Research and Evaluation (AGREE-II) instrument was used to assess the quality of breast cancer and sarcoma guidelines and protocols according to six independent domains: Scope and Purpose, Stakeholder Involvement, Rigour of Development, Clarity of Presentation, Applicability, Editorial Independence, and a general question rating overall quality. Three independent appraisers rated a total of 11 guidelines and 10 protocols from 8 websites developed for health care settings in North America, the United Kingdom, Europe and Australia.
Results: Mean quality scores across domains were highly variable, ranging from 29-73% for guidelines and 31-71% for protocols. Guidelines scored highly in terms of articulating their scope and purpose (72.6±11.2%) but scored poorly with respect to their applicability in clinical practice (29.0±17.3%). Cancer Care Ontario (CCO) and National Institute of Clinical Excellence (NICE) guidelines achieved the highest scores across most domains. Protocols scored highly on clarity of presentation (70.6±17.6%) but poorly in terms of the processes used to synthesize the underlying evidence and formulating and updating recommendations (30.8±20.0%). CCO and eviQ protocols achieved the highest scores across most domains. We did not find any differences in the quality of breast versus sarcoma guidelines/protocols and a basic navigability assessment revealed that resources were generally easy to locate.
Conclusions: Overall, the quality of resources was modest and most websites fared well in relation to factors associated with successful uptake of computerized clinical support tools by clinicians. Our evaluation provides a quick reference tool for clinicians about the strengths and limitations of oncology resources across several major websites. Further, it supports resource developers in terms of where to direct efforts to enhance guideline and protocol development processes or the way in which they communicate these processes to end-users.
See more of: The 33rd Annual Meeting of the Society for Medical Decision Making