##
SPEEDING UP THE SIMULATION CALIBRATION PROCESS USING MACHINE LEARNING METHODS

** Purpose: ** Model calibration to estimate unobservable model input
parameters often involves evaluating large numbers of candidate parameter sets
to find suitable values. As such, it can be a laborious and time-consuming
activity. Our objective was to improve the calibration process by utilizing
machine learning tools such as artificial neural networks (ANN). ANNs are
computational algorithms shown to be successful in prediction and coupled with
active learning processes are ideally suited to efficiently identify promising
candidate model input parameter sets.

** Methods: ** As our test bed, we used a representative calibration
process for a breast cancer simulation model that involved examination of 378,000
input parameter sets from which 69 were considered to produce good model fit. We
first evaluated a random subset of parameter sets and constructed an ANN model
to predict which of all possible sets are more likely to generate desired
outputs. We further improved the predictive accuracy of the ANN model by using
an

*active learning*approach, where we do the model training iteratively by selecting a small number of input vectors to evaluate by the simulation at every iteration. Active learning allowed us to start with a smaller number of initial simulation runs to form our training set and gradually enlarge the training set by choosing the most promising parameter sets.

** Results: ** We initially evaluated 2000 parameter sets and built our
ANN model based on this training set. Using the active learning approach, we found
all 69 good fitting parameter sets by evaluating only 4500 of the 378,000 combinations.
Compared to the active learning approach, our initial ANN model required
evaluating more than 15000 input vectors. Figure 1 shows number of good vectors
found by ANN model only and ANN coupled with active learning as number of
evaluations increase.

** Conclusions: ** For many simulation models using calibration, evaluating
all parameter combinations is prohibitive. Machine learning methods can guide
model developers for selecting more promising parameter combinations and hence
speeding up the calibration process. Our tests on a previously developed breast
cancer simulation model showed that evaluating only 1.2% of all combinations
would be sufficient for the calibration of this model.

Figure SEQ Figure \* ARABIC 1:
Active Learning vs ANN

See more of: The 36th Annual Meeting of the Society for Medical Decision Making