Purpose:
We propose methodology for developing and validating decision rules which recommend one of two or more competing treatments on the basis of minimizing expected risk of an outcome of interest.
Methods:
We present our methodology in the setting of recommending either coronary artery bypass graft (CABG) or percutaneous coronary angioplasty (PCA) for individual patients on the basis of minimizing risk of in-hospital mortality.
Data on 406,456 inpatient discharges for which the primary procedure was either CABG or PCA were extracted from seven AHRQ State Inpatient Databases (2009-2010). Generally, our strategy was to simultaneously develop two models for the probability of mortality under each respective treatment using age, gender, and the set of (hierarchically-aggregated, based on cell size) present-on-admission diagnosis codes, and then define the decision rule as: Recommend CABG if the probability under the CABG model is lower than the probability under the PCA model, and recommend PCA otherwise.
Data were randomly partitioned into separate training and model calibration datasets for each treatment (50% and 25% of each treatment's samples, respectively) as well as a combined test dataset (25% of each treatment's sample) for evaluating the decision rule against existing practice (see Figure).
Elastic net regression was used to fit parsimonious models to their respective training datasets; each model was then recalibrated using the respective model calibration datasets so that predicted probabilities better represented observed outcome incidences in the test dataset. The decision rule was then defined according to the recalibrated models' probability estimates for a given set of comorbidity values.
The test dataset was then used to recommend either treatment on the basis of the decision rule. We then compared the odds of mortality between discharges conforming and not conforming to the rule's recommendation.
Results:
In the test dataset, mortality was observed for 365/22,664 (1.6%) CABG discharges and 793/79,079 (1.0%) PCA discharges. Modeling resulted in 18 and 81 predictors for the CABG and PCA models, respectively. Overall, there were 27,572/101,743 (27%) non-conforming discharges: 6,025/7,142 (84.4%) recommended for CABG and 21,547/94,601 (22.8%) recommended for PCA. Odds of mortality were 4.51 times greater among non-conforming discharges than among conforming discharges (95% CI: 4.01-5.09).
Conclusion:
This approach may be effective for informing common decisions. Future research will refine the methods, include preferences, and define limitations.
