A COMPARISON OF LIKERT AND BEST-WORST SCALING FOR MEASURING PATIENT PRIORITIES: A RANDOMIZED CONTROLLED TRIAL

Purpose: Researchers, policy makers, and clinicians are increasingly interested in measuring the priorities of patients. Simple Likert items, distinct from Likert, are used to assess priorities using a simple categorical scale to assess if patients view concepts positively, neutrally, or negatively. Best-worst scaling (BWS) Case 1 can also be used to measure priorities and can be especially beneficial when there is a need to value concepts both positively and negatively. We sought to compare Likert items and BWS for assessing the barriers and facilitators to self management among patients with type II diabetes.

Method: A randomized controlled trial (ClinicalTrials.gov Identifier: NCT02637609) was conducted among a nationally representative, and racially/ethnically diverse, sample of patients with type 2 diabetes in the United States. Respondents evaluated 11 concepts that were purposively selected through a robust mixed-methods environmental scan to represent both known barriers and facilitators of patients’ diabetes management and were randomized to either Likert or BWS. Standardized mean scores were calculated for each object based on the Likert and BWS methods and the results were compared graphically and tested statistically via Spearman’s Rho. We also compared measures of respondents’ understanding of the tasks and associated burdens.

Result: Randomization to Likert (n=549) and BWS (n=554) resulted in balance in respondent characteristics. While the results were highly correlated across the methods (Spearman’s Rho=0.973), respondents in the Likert arm did not value any factor as a barrier, while BWS identified concepts that respondents valued both positively and negatively. BWS also had tighter standard errors across all objects. Despite the statistical problems associated with their use, respondents considered Likert items both easier to understand, easier to answer and more reflective of their preferences (P<0.01).

Conclusion: Despite the high correlation between the results, BWS seems to have numerous advantages. Respondents tended use only the positive responses on the Likert items, despite considering some concepts that have been documented as barriers in the literature. However, respondents found Likert items less burdensome – both given the novelty of BWS and its relative burden.  This said, while the burden for each respondent is less, many more respondents are needs and even then results may be biased. Moving forward, researchers, policy makers, and clinicians should consider BWS an alternative to simple Likert items.