Purpose: Pragmatic trials result in rich data sets often with large heterogeneity between subjects both in background variables and in response. We take advantage of this heterogeneity to develop an exploratory analysis for reliably identifying possible interactions between patient variables and optimal treatment, and thereby suggest avenues for individualization of treatment in future research.
Method: We use a bootstrap aggregation procedure to estimate the "selection probability" for each treatment---the probability that the treatment would be found optimal in a repeated trial and analysis. This estimated selection probability serves as a measure of confidence in the optimality of each treatment. We discuss why traditional measures of confidence such as p-values are not useful in this context, particularly when comparing more than two treatments simultaneously. We assess the performance of the bootstrap aggregation method using simulation studies. Finally, we apply the method to data from the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial, wherein the most recent self-reported Quick Inventory of Depressive Symptoms (QIDS) score is used to predict the optimal treatment for individuals.
Results: Our simulations demonstrate how the bootstrap aggregation confidence measure can be used to indicate possible interactions between patient variables and optimal treatment while mitigating against spurious "false positives." Furthermore, the example using the STAR*D data shows how the results can be presented in a way that is intuitively interpretable by a wide audience.
Conclusion: This bootstrap aggregation exploratory analysis, combined with pragmatic trial data, yields useful clinical insights that can suggest avenues for individualization of treatment in future research.
Candidate for the Lee B. Lusted Student Prize Competition