EXAMINING THE IMPORTANCE OF UNCERTAINTY AROUND AMERICAN COLLEGE OF MEDICAL GENETICS (ACMG) RECOMMENDATIONS FOR NEWBORN SCREENING FOR CITRULLENIMIA TYPE 2 (CIT II)

Purpose: In 2006 the American College of Medical Genetics (ACMG) published recommendations for newborn screening (NBS) that appear largely responsible for subsequent NBS expansion. ACMG ignored uncertainty surrounding its process’s foundation of survey responses: potential impact of missing data and of sampling variability for its mean survey scores that determined an entry point to an algorithm (EPA) that determined its final recommendations.  We examine one condition –Citrullinemia Type II (CIT II) and 2 of 18 survey questions, those for which responses were reported.

Method: ACMG implicitly used an “invariance assumption” - that expected values of missing responses equaled those of observed. That enabled “identification” of the mean estimate, but with an assumption of unknown credibility.  We assessed the impact of missing data by using a boundary estimate (Manski, 1989) procedure whereby best and worst case scoring was assigned to missing data. The bounds on the scores for the missing observations in turn implied bounds for the mean scores.  This enables only “partial identification” – but without making assumptions about the nature of the missing data. We also bootstrapped the original data to assess sampling variability around mean scores.  Then we combined these analyses by bootstrapping around the boundary estimates.  For all results we assessed whether the range of possible scores implied by the uncertainty uncovered was sufficient to alter the EPA and the recommendation.

Result: We report the combined analysis of both variables bootstrapped around the original mean and around its lower bound.  When bootstrapping around the original mean, 46% of the mean values implied a lower EPA.  When bootstrapping around the lower bound, that percentage increased to 84%.  There appears to be little confidence that the original EPA is the correct one.  A lower EPA implied different questions in the algorithm. However, in the case of CIT II, it is clear from these questions that the recommendation could not change - regardless of the score.

Conclusion: Despite uncovering considerable uncertainty around the scoring and EPA, the CIT II NBS recommendation could not change. This robustness to the range of possible score changes – indeed any score changes – appears to call into question the validity of the ACMG exercise, the results to which - for this condition – are insensitive to all the survey responses it assessed.