DO SURGERY JOURNALS REPORT RISK REDUCTION INFORMATION IN A WAY THAT COULD PROMOTE BIAS?

Purpose: Denominator neglect is the tendency to focus on the numerator in a ratio (e.g., number of affected individuals) and ignore the denominator (e.g., overall number of people at risk). This effect can result in inaccurate perceptions of effectiveness of medical interventions and treatments when health professionals compare groups of patients of unequal sizes. We investigated whether the way that data are reported in medical journals would promote denominator neglect. In particular, we investigated (1) to what extent ratio comparisons giving rise to denominator neglect are prevalent in the medical literature, focusing on orthopedic surgery,  and (2) in what way the data are reported.

Method(s): We downloaded all abstracts of original research articles published every even (vs. odd) year in the past 10 years in seven leading orthopedic surgery journals (N=9,887). We randomly selected 25% of abstracts of each journal for review (N=2,472). The final sample consisted of 405 articles that reported relevant ratio data.

Result(s): The majority of articles (365, 90%) reported unequal group sizes. Of these, 233 (64%) reported both the number of affected individuals and the associated percentage of the total. However, 67 (18%) reported the number of affected individuals only, and only 24 (36%) of those reported the denominator alongside. Articles published earlier, reports of randomized controlled trials, and articles published in journals with higher impact factor were more likely to report the number of affected individuals only.

Conclusion(s): A significant proportion of articles report only the number of affected individuals in each group, without facilitating group comparison by specifying the percentage alongside. Paradoxically, this reporting was more prevalent in articles documenting high quality evidence. These results suggest that data are often reported in ways that could make readers susceptible to denominator neglect, resulting in inaccurate perceptions of treatment effectiveness or risk reduction. We are currently investigating to what extent medical professionals are susceptible to denominator neglect compared to the general population and if the effects of denominator neglect persist when the percentages are reported alongside the number of affected individuals.