Background: Decision analysis modeling (DM) is commonly employed for evaluations of heath care technologies and is especially informative when conducting a randomized clinical trial (RCT) is not feasible. RCTs are considered the “gold standard” for evaluating therapeutic interventions. However, it is not known how often results from DM and RCT either concur or differ.
Objective: To conduct a systematic review to compare the findings of DM with matching RCTs.
Methods: We searched PubMed from 2000 to 2008 using the following Mesh and text terms: ("Decision Support Techniques"[Mesh] OR ("Decision Making"[Mesh]) OR ("Decision Analysis") AND ("Therapeutics"[Mesh] OR "therapy "[Subheading] OR "Treatment Outcome"[Mesh] OR "Therapies, Investigational"[Mesh])). No limits were used. Abstracts of all the identified studies were reviewed by two investigators and only studies comparing two or more therapeutic strategies were included in the review. Subsequent RCTs matching DM studies were identified using search strategy described by Haines et.al (2005). A RCT was considered a match with DM if it enrolled a similar population, evaluated similar interventions and outcomes and drew the same conclusions as DM. Two authors jointly ascertained the agreement or disagreement between the outcomes of the DM and matching RCT.
Results: The initial search yielded 25,000 studies. Eighty seven DM studies pertaining to 63 diseases were included. Breast carcinoma was the most studied disease (8/87 studies).Out of 87 studies we found 48 matching RCTs (Figure 1). In 52% (25/48) of instances there was an agreement and 48% (23/48) of cases there was disagreement between the conclusions of decision analyses and subsequent RCT.
Conclusions: The findings from our study show that decision analysis modeling disagrees with randomized clinical trial in about 50% of time.
Figure 1