Purpose: For any disease modeling, calibration of the simulation output to some existing clinical data is vital to the validation and the prediction of the computer model. Calibrations often involve 1) defining goodness of fit to multiple targets and 2) searching for the optimum parameter set (i.e., yields the best fit). Currently, few multicriteria optimization techniques, which are common in engineering and economics, are utilized by health policy researchers. Here we report on our experience adapting one ‘analogy-based' and one ‘evolutionary' search algorithm to disease modeling.

Methods: We calibrated our microsimulation Lung Cancer Policy Model to SEER data, using a weighted sum method of multiple targets to quantify goodness of fit. Targets were classified and weighted by importance, noise level, and relative magnitude. We compared the efficiency of two advanced search algorithms, simulated annealing (SA) and a genetic algorithm (GA), to that of a previously conducted grid search of the parameter space. Efficiency was defined as the number of parameter sets tested before converging to a small tolerance range of the grid search optimum. ‘Tuning' parameters, intrinsic to SA and GA, were varied to assess their effects on efficiency. For SA, the ‘Tuning' parameters include the artificial temperature (T) and the annealing step (DT). For GA, the ‘Tuning' parameters include the size of each generation (N), the reproduction probability (PrR), and the mutation rate (PrM).

Results: Both SA and GA located the optimum in <2,000 parameter sets, versus >50,000 sets tested in the original grid search. For serial simulations, SA consistently outperformed GA in efficiency. Empirically, we confirmed that a 50% acceptance probability for SA assures a reasonable sampling rate in the annealing schedule. Giving less weight to intrinsically noisy targets (or eliminating them from the minimization criteria) improved the efficiency of SA up to 200%.

Conclusion: Many of the advanced optimization techniques common in engineering and economics are applicable and feasible to implement in disease modeling. By incorporating the advanced optimization algorithms into our Lung Cancer Policy Model, we observed a roughly 25-fold gain in efficiency compared to previously conducted grid search.