DETERMINATION OF HOSPITAL PERFORMANCE ON THE BASIS OF “ELIGIBLE” PATIENTS

Purpose: With the initiative of pay-for-performance, there is an increasing movement towards reporting of hospital “performance cards” for clinical process measures. Patients, professional societies, and purchasers of health care increasingly rely on these data to assess quality of care. The report cards typically include the number of eligible patients for standard therapies and the number of patients receiving therapies among the number eligible for each therapy, but not the number of patients admitted. We studied the ability of the process-based performance measures, constructed from the numbers of treated and eligible either as the raw ratio of the two or as the estimates derived from a hierarchical Bayesian model, to measure individual hospital quality and to classify the hospitals into tiers indexing distinct quality.

 

Method: Through Monte Carlo simulations, we generated “true” quality data using a two-part model in which the first part characterizes the inclusion probability for being eligible conditional on patient admission, and the second part characterizes the probability for receipts of necessary therapy conditional on patient eligibility. The parameters for simulations are solicited from 2006 Hospital Compare national database. We compared the performance among the conventional ratio estimates and the hierarchical Bayesian estimates based on the second part of the model as well as those based on the full model. We contrasted sensitivity, specificity, and the area under the receiver operator characteristic curve (AUC) under several different assumptions.

 

Results: The hierarchical Bayesian estimates obtained from the second part of the model perform very similar to and at most slightly worse than those derived from the full model. The closeness of the two is quite stable across reasonable ranges of hospital sample size, the correlation between the inclusion probabilities in the two parts of the model, and treatment rates. The ratio estimates are modestly worse than the hierarchical Bayesian estimates in classifying hospitals into different tiers (e.g., the AUC of the former is generally 4-7% lower than that of the latter). But the sensitivity/specificity of the ratio estimates can be either higher or lower than that of the hierarchical Bayesian estimates, depending on the distribution of treatment rates.

 

Conclusion: Estimates of the quality of treatments can be adequately obtained based on eligible patients.