SEARCHING FOR STUDIES BEYOND PUBMED: WHAT IS THE BENEFIT OF SEARCHING MULTIPLE DATABASES?

Purpose: Search of multiple electronic databases when conducting systematic reviews of the published literature to support health technology assessmentis is considered good practice. The US National Library of Medicine’s MEDLINE (accessed through PubMed) covers clinical medicine and the health sciences. We aimed to investigate the incremental yield of searching databases beyond PubMed in systematic reviews of therapeutic interventions. 

Method: From the Cochrane Library (Issue 4, 2012) we randomly selected 50 reviews that searched the Pubmed/MEDLINE and EMBASE databases and meta-analyzed at least 10 studies. For each eligible citation in each review, we checked whether it was retrievable in PubMed and EMBASE. For the first-listed meta-analysis of 10 or more studies in each review we examined whether excluding studies not found in PubMed or in EMBASE changes summary estimates.

Result: The 50 Cochrane reviews included a median of 39 papers (25^th-75^thpercentile: 19-75). The median numbers of PubMed-, EMBASE-, and non-PubMed/non-EMBASE-indexed papers were 29 (17-53), 33 (17-57), and 4 (1-9), respectively. Searching EMBASE in addition to PubMed contributed a median of 1 (0-3) more papers. The selected meta-analyses included a median of 14 studies (12-19), of which a median of 12 (10-16) were indexed in PubMed and 13 (11-17) in EMBASE. A median of 0 (0-1) studies per meta-analysis were identified through EMBASE but not PubMed. Relying only on PubMed would change the statistical significance in 1 out of 50 meta-analyses compared to using all studies (P=0.024 vs. P=0.068, respectively). In meta-regressions PubMed-indexed studies had a significantly different effect size compared to non-PubMed-indexed studies in only 2 datasets.

Conclusion: The gains from searching sources beyond PubMed, and from searching EMBASE in particular, for systematic reviews, may be more modest than commonly believed.