AN EFFICIENT HYBRID CALIBRATION APPROACH COMBINING ARTIFICIAL NEURAL NETWORK METAMODELING AND BAYESIAN METHODS

Purpose: Bayesian methods  are naturally suited for calibration because they reveal the posterior distributions of the model parameters and their correlations, unlike direct search algorithms [e.g., Nelder-Mead (NM)] that only produce point estimates.  However, Bayesian methods are rarely implemented in practice due to technical and computational challenges associated with defining simulation models in specialized software (e.g., BUGS).  We propose combining artificial neural network (ANN) metamodeling with Bayesian calibration as a hybrid approach that is efficient and can be quickly scaled to models of arbitrary complexity. 

Methods: Our approach involves these steps: (1) conduct a PSA with vague input parameter values, (2) fit an ANN metamodel using the PSA's inputs and outputs, (3) calibrate the ANN metamodel using Markov chain Monte Carlo (MCMC) sampling algorithm, and (4) obtain the posterior distribution of the calibrated parameters that quantifies all sources of uncertainty not explained by the simulation model or the observed data. We demonstrate our approach with a Markov model for cancer progression. The model has three states: Cancer free, Metastasis and Dead, and two unknown probabilities that define the transitions between cancer free and metastasis (pMet) and death from metastasis (pDieMet). We produced 100 survival curves from the Markov model using 100 random parameter sets for pMet and pDieMet. We compared the accuracy of the Hybrid approach to estimate the true parameter values for each parameter set relative to NM calibration. In addition, we initialized  NM from 100 random starting points, while we initialized the Hybrid approach from a single starting point.

Results: The Hybrid approach was more precise than NM.  The mean squared errors for NM were more than 1000 times and 200 times greater than the Hybrid approach for pMet and pDieMet, respectively.  The Hybrid approach took 2.9 seconds compared to 1.6 seconds for NM. The Figure shows the results of the Hybrid approach for one set of true parameter values indicated by the star (pMet=0.0099 and pDieMet=0.0477).  In addition the Hybrid approach reveals the posterior distribution and the correlation between pMet and pDieMet, which are not possible with direct search algorithms like NM.

Conclusions: Bayesian calibration reveals posterior parameter distributions and their correlations for calibrated model parameters.  Adding ANN metamodeling can overcome many technical and computational challenges associated with Bayesian calibration.