A MULTIFACETED LONGITUDINAL COMPARATIVE ASSESSMENT OF EVIDENCE-BASED MEDICINE SKILLS

Purpose: To examine the longitudinal impact of EBM instruction on skills (formulating a clinical question, appraising validity of research, revising decisions based on evidence), behaviors (use of an EBM librarian consult service), and attitudes, in contemporaneous groups of Pediatrics residents, Family Medicine residents, and Health Science Librarians.

Methods: A 2x4 (Pediatrics vs. Family Medicine residents x time period) mixed model repeated measures design was used, along with a comparison to a group of 8 librarians during the same period. Over the course of two years, 106 residents (PGY1-3) in two programs at the same University were assessed semiannually by a question formulation task (4 items), a critical appraisal task (60 items, Cronbachs =0.71), a decision task (4 items on a factorial design), and an EBM attitude inventory (25 items, =0.79). Linear mixed models were fitted to control for missing data and intrasubject correlation. EBM education in Pediatrics was considerably more intense and had greater focus on the development of clinical questions than education in Family Medicine at this University (a weekly EBM conference vs. a monthly journal club).

Results: At baseline, performance did not differ significantly between the programs on any measures. At the second measurement, Pediatrics residents performed significantly better at validity appraisal than Family Medicine residents (26.9 vs. 20.6, p<.05), but this difference was attenuated by the fourth session. Strikingly, the Family Medicine residents performance at question formulation degraded over time, eventually resulting in significantly worse performance than the Pediatrics residents (F(3,57)=4.7, p=.005). There were no differences in the decision task or attitude toward EBM. Year of resident had no effect, and the performance of librarians was similar to (and in some cases better than) Pediatrics residents.

Conclusions: This multifaceted longitudinal assessment was capable of identifying differences in the EBM preparation of residents across time and between two programs and reflected differences in the instruction in the programs. The results, and particularly the strong performance of the librarians, suggest that EBM skills are both learnable and not automatically acquired through clinical practice.