CONSUMER USABILITY TESTING OF A WEB-BASED PATIENT REPORTED OUTCOMES (PROS) AND DECISION SUPPORT TOOL (MYPSYCKES): IMPORTANT LESSONS FOR PROS SYSTEMS

Purpose: Integrating patient reported outcomes (PROs) into clinical work is a promising method for promoting more patient centered care. Web-based tools have the capacity to routinely capture PROs, but little data is available on how well individuals with serious mental illness and other vulnerabilities respond to and use these technologies.  MyPSYCKES, a bilingual web-based application developed by the New York State Office of Mental Health (NYS OMH), facilitates the use of PROs in the outpatient setting. We conducted usability testing of MyPSYCKES among users at three diverse outpatient clinics in New York State.

Method: We recruited subjects from three clinics that were currently implementing the MyPSYCKES program.  During each usability testing, subjects completed a MyPSYCKES survey online where they answered structured and personalized questions about their use of wellness activities and medications, symptoms and status, concerns about medication, and goals for treatment. The usability testing followed a Think Aloud protocol, asking subjects to speak their thoughts and impressions as they navigated the program.  The testing sessions were audio and video recorded, and structured fieldnotes were used to capture subject task completion and feedback. 

Result: 7 subjects completed the usability testing of MyPSYCKES.  5 subjects completed the MyPSYCKES survey and report without assistance.  When given the choice, 2 subjects preferred the touchscreen feature over the mouse, and 1 subject preferred the voiceover feature over reading the text. Clients found all survey questions understandable and useful, however personalized questions had less face validity than standardized questions. Dominant themes from subjects’ Think Aloud narrations include: difficulty anchoring responses in time-based answer options, how routine outcome reporting provides insight on progress, the usefulness of the tool for clinical communication, and increasing opportunities to report on life circumstances. 

Conclusion: These data from our initial round of usability testing provide important feedback on the value and challenges of electronic patient reported outcomes systems, as well as the feasibility of the Think Aloud method for this population.  While our sample was small, most subjects with serious mental illnesses were able to use the web-based program to complete a report of their outcomes independently, but their feedback suggests that patient reported measures should provide more opportunities for patients to record context and anchor their responses to their individual experiences.