DEVELOPMENT OF AN ITEM RESPONSE THEORY-DERIVED ADAPTIVE SHORT FORM FOR OUTCOME ASSESSMENT

Background: In health care settings where computer adaptive testing (CAT) is not feasible, adaptive short forms (ASF) provide efficient outcome measurement capability. Using IRT methods, a calibrated collection of items representing a range of difficulty for a particular outcome dimension is created (i.e., an item bank).  An ASF consists of items drawn from the calibrated item bank, providing the basis for linking different setting-specific instruments along a common outcome scale, and creating a bridge to outcome monitoring using CAT applications.  

Purpose: To develop an ASF for the Life Skills (LS) domain of the Activity Measure for Post-Acute Care (AM-PAC) item bank for outpatient rehabilitation settings.                                                          

Methods: We used IRT methods to identify optimal items for the ASF from the AM-PAC item bank using data from a convenience sample of 338 rehabilitation patients with neurologic disorders.  

We used test information function (TIF), conditional reliability, and item exposure rate (IER) on the CAT to identify items for the ASF. The continuous item information values were summed to form TIFs.  We  identified a subset of items that best fit  precision levels needed to match the person ability scores.  When the TIF peaked around the same range on the scale as the patients’ peak of ability distribution, the instrument was assumed to “fit” the sample. The IER, the ratio of total number of times an item was administered over total number of test occasions in a study, identified which AM-PAC items were administered more often in the application. Items with the best discrimination and item information typically have high IERs.  We calculated the correlation, mean residual and root mean square residual (RMSR) of the score based on the ASF and those based on full item bank. We  selected the best19 items from the LS item bank and calculated the same statistics. We calculated ceiling and floor effects for the final ASF and the full item bank.

Results: The correlation for ASF with the item bank was 0.94243(92^nd percentile); mean residual: -0.1331163(100^th percentile); RMSR: 0.55(4.1^st percentile). No floor effects were noted, and ceiling effects were 22.5% for the ASF compared to 13.6% or the full item bank.

Conclusion:  The LSs ASF demonstrated acceptable psychometric properties and provides a useful tool for outcome measurement when a CAT is not feasible.