This presentation is part of: Poster Session - Clinical Strategies; Judgment and Decison Making
VALIDATING AND UPDATING A PREDICTION RULE FOR NEUROLOGICAL SEQUELAE AFTER CHILDHOOD BACTERIAL MENININGITIS
C.J. Biesheuvel, MSc, Y. Vergouwe, PhD, D.E. Grobbee, PhD, and K.G.M. Moons, PhD. University Medical Center Utrecht, Julius Center for Health Sciences and Primary Care, Utrecht, Netherlands
Aim Recently, a prediction rule for developing neurological sequelae after childhood bacterial meningitis was developed on a sample of 170 patients derived from two pediatric teaching hospitals. Before implementing in practice, a rule must be tested in new patients (external validation). Our aim was to study the external validity and, if necessary, to update this rule. Methods The original prediction rule was developed using multivariate logistic regression analysis and included gender, atypical convulsions in patient history, body temperature at physical examination and type of pathogen. In order to validate this rule, it was applied to 628 patients, collected from almost all hospitals in The Netherlands (validation set), and the probability of neurological sequelae was estimated for each patient. We assessed calibration with a calibration line that described the relation between the predicted probabilities and the observed frequencies, and with the Hosmer-Lemeshow goodness-of-fit test. The discriminative ability was studied with the ROC area. Finally, we updated the original rule by adding extra predictors and re-estimating the regression coefficients using the merged derivation and validation sets. Results The calibration plot of the original rule in the validation set showed poor agreement between predicted probabilities and observed frequencies for the patients of the validation set. This was confirmed by a significant Hosmer-Lemeshow test (p-value< 0.01). The ROC area was 0.65 (95%CI: 0.57-0.72), which was significantly lower than the area found in the derivation set (0.87 (95%CI: 0.78-0.96)). In the merged data sets, gender was no longer an important predictor. In addition to atypical convulsions, body temperature at physical examination, and type of pathogen, the use of anti-epileptica > 2 days during hospital admission, and presence of petechiae and/or ecchymoses appeared to be important predictors of neurological sequelae. The ROC area of this updated rule was 0.75 (95%CI: 0.67-0.83) after correction for overoptimism and its calibration was acceptable. Conclusion The former developed prediction rule for neurological sequelae after childhood bacterial meningitis showed poor performance when applied to another much larger population. The updated prediction rule showed adequate calibration and discrimination in the merged data sets. Further study of model validity may stimulate application in clinical practice.
