Purpose: Increasingly microsimulation models are used to evaluate cancer screening programs. A model's prediction of screening effectiveness depends on the values of unobservable natural history parameters. Calibration is the process of inferring the values of unobservable parameters by constraining model output to replicate observed data. Because there are many approaches for model calibration and little consensus on best practices, we surveyed the literature to catalogue the use and reporting of these methods in cancer screening models.

Methods: We conducted a MEDLINE search (1980 through 2006) for articles on cancer screening models and supplemented the search results with personal references. For each article, two authors independently abstracted pre-determined items using a standard form. Data items included cancer site, type of simulation model, methods used for determination of unobservable parameter values, and description of any calibration protocol. All authors reached consensus on items of disagreement. Reviews and non-cancer models were excluded. Articles describing analytical models which estimate parameters with statistical approaches (e.g., maximum likelihood) and articles describing models that did not incorporate the underlying natural history (e.g., stage-shift models and simple decision analyses) were catalogued separately. Models that included unobservable parameters were analyzed and classified by whether calibration methods were reported and if so, the methods used.

Results: The MEDLINE search yielded 134 unique articles; 20 of these were excluded on the basis of the title/abstract. The remaining 114 articles were supplemented with 24 articles from personal reference lists, for a total of 138 articles read and abstracted. 101 articles reported on models that incorporated the underlying natural history of cancer and therefore required calibration. Of these, 65 mentioned model calibration (explicitly or implicitly) and provided at least some detail on calibration methods in the article itself or in a referenced article. 27 papers reported the calibration goodness of fit (GOF) metric in the paper itself, and 8 of these described the algorithm used to search the parameter space. A visual-fitting approach was the most commonly reported GOF metric.

Conclusions: Thorough descriptions of calibration procedures are rare in the published literature on cancer screening models. Calibration is a necessary component of model development and is central to the validity and credibility of subsequent analyses and inferences drawn from model predictions. Transparent reporting of methodology is critical.