Background: Decision and cost-effectiveness analytic models require estimates of treatment effectiveness as proportion of patients achieving positive treatment outcomes (responder analysis). This approach often means dichotomizing continuous scores into thresholds of positive/negative response. Several approaches for calculating these thresholds exist, but have not been directly compared nor validated.

Purpose: The purpose of this work was to test the validity of different threshold approaches against indicators of patient recovery .

Methods: Secondary analysis of a cohort of patients undergoing physiotherapy for soft tissue shoulder disorders followed for 12 weeks or until discharge, using the Disabilities of the Arm, Shoulder and Hand (DASH) (n=272). Response thresholds were defined by minimal clinically important change, normative or low-disease scores, or a combination of the two (change plus achievement of a healthy state). Individual patients were assigned to having met (“improved”) or not met each criterion. Two indicators of recovery were used for validation: a) meeting treatment goals, and b) self rated change in shoulder problem (9-10/10, where 10=much better, 0=much worse).

Analysis: Proportion improved according to each threshold were calculated and compared. Sensitivity and specificity of each threshold to each indicator of recovery were calculated. The one with highest sensitivity and specificity was considered most useful.

Results: Proportion considered ‘improved' ranged from 6 to 89% across approaches. Minimal change thresholds were sensitive (>0.81) but less specific (~0.2-0.4). The combined approach, particularly using normative data for a healthy state, had a balance of sensitivity and specificity (0.66, 0.82 respectively).

Conclusions: Different approaches led to different thresholds and very different proportions improved. Change thresholds lacked specificity (high false negative rate) wihle the combined threshold seemed the most valid. Given the lack of a single, agreed upon threshold, decision and cost-effectiveness analysts should test the sensitivity of their results to various approaches for defining effectiveness through sensitivity analyses.