METHODS OF DISEASE MODEL CALIBRATION: THEORY AND PRACTICE

Format Requirements: The format of this course combines lecture-style presentations with hands-on spreadsheet exercises to introduce and build on theoretical and practical aspects of model calibration. The initial third of the course reviews the conceptual foundations of disease modeling (briefly), situations where model calibration is useful, and the main phases of carrying out model calibration. The second third of the course utilizes hands-on spreadsheet exercises to walk participants through various calibration parameter search and evaluation approaches. The final third of the course introduces advanced topics and related spreadsheet demonstrations. Proficiency with Microsoft Excel is a required, and familiarity with Excel macros/Visual Basic for Applications (VBA) programming and/or Solver Add-in is preferred but not required. This course is taught at an intermediate level and is intended for those with experience or interest in programming or evaluating disease models. Participants should bring laptop computers for the interactive exercises.

In the context of disease modeling, “calibration” refers to the systematic adjustment of model inputs such that the resulting model outputs better reflect setting-specific, observed disease outcomes. This course encompasses the main theoretical and practical aspects of model calibration methodology. Stylized practice models and realistic oncology disease models programmed in Microsoft Excel are utilized to give participants hands-on experience employing commonly-used and cutting edge model calibration techniques. The main objectives of the course are: 

• To review modeling circumstances that particularly benefit from calibration;
• To introduce the main methodological phases of model calibration, specifically: 1) Identifying calibration inputs/outputs;  2) Determining goodness of fit criteria, such as windows-based targets, minimizing deviation/least-squares approaches, and likelihood-based functions;  3) Selecting and implementing parameter search algorithms, including manual adjustment of model parameters, random searches, and optimization techniques (linear programming and directed-search algorithms);
• To demonstrate how to implement these methods using interactive spreadsheet exercises;
• To highlight several advanced topics.

Specific advanced topics include:

• Comparisons of manually-intensive versus computationally-intensive parameter search strategies (such as simulated annealing, Latin hypercube, and Nelder-Mead search algorithms);
• Identification of and correction for bias introduced from calibrating longitudinal models to cross-sectional data;
• Probabilistic and deterministic uncertainty analysis for calibrated disease models;
• Bayesian approaches to model calibration.