Purpose: An increasing number of decision models are calibrated to external datasets, but calibration practice is highly heterogeneous. We decomposed a published calibration approach into 7 steps, optimized each step, and reassembled them for routine use. The approach was applied to a Markov model of pressure ulcer (PU) history in nursing home. The aim is to calibrate 8 progression and healing probability estimates (PEs) of PU stages 1-4 in order to reproduce age- and stage-specific prevalence from the same Minimum Data Set (n=18,321).
Methods: The approach consists of 7 steps: 1) Parameters: 8 uniformly distributed multipliers were used to characterize the uncertainty in the stage-specific PEs. 2) Targets: observed and projected prevalence data were to be compared using Pearson’s goodness-of-fit statistic; 3) Sampling: Latin Hypercube (LH) samples of calibration parameters were generated first with independence assumption then with an estimated correlation structure (step 4). 4) Model evaluation: candidate models were evaluated, prevalence data projected; fit statistics derived; good-fit models identified; and correlation structure among calibrated model estimated. 5) Regression analysis: was used to correlate smoothing functions of the calibration parameters with fit statistics. 6) Good-fit models: final good-fit models were selected after repeating steps 1-4. Calibrated parameters were displayed using star plots to identify clusters and outliers among the equifinal models. 7) Projections: expected costs and QALYs comparing pressure-redistribution mattresses (PRMs) and standard mattresses (SMs) for PU prevention were generated.
Results: The characterization of uncertainty in the PEs was inspected via the smoothing function plots; ranges of 4 uniform distributions were extended because the smoothing functions displayed truncated trends of decreasing fit-statistic. In 3-D scatter-plots, the good-fit region defined by calibrated parameters appeared to be connected but not easily characterized, except via their correlation structure. Correlated LHS samples yielded 110% more good-fit models compared to samples with independence assumption. The second 10,000 correlated LH samples yielded 973 good-fit models with substantial variation in calibrated parameter compositions. Projections from the good-fit models showed the PRM option to consistently improve QALYs with widely variation in expected incremental cost.
Conclusions: The systematic approach outlines the practical steps for routine calibration. Its use facilitates HTAs of preventive interventions in nursing homes, hospitals and home care. This is an evolving field and further research is needed.
Candidate for the Lee B. Lusted Student Prize Competition