CORRECTING FOR PARTIAL VERIFICATION BIAS: A COMPARISON OF METHODS

Purpose: a common problem in diagnostic research is that the reference standard has not been performed in all patients. This partial verification may lead to biased accuracy measures of the test under study. Several solutions have been proposed to alleviate this bias. The authors studied the performance of multiple imputation and the conventional correction method proposed by Begg and Greenes under a range of different situations of partial verification, to examine under which circumstances they produce similar results and when their results differ.

Method: in a series of simulations, using a previously published Deep Venous Thrombosis dataset (N=1292), the authors deliberately set the outcome of the reference standard to missing based on various underlying mechanisms and by varying the total number of missing values. They then compared the performance of different correction methods (ie Multiple Imputation and the Begg and Greenes correction method) in each of these patterns of verification, in particular their ability to reduce the bias in estimates of accuracy by comparing it with the true value in the complete dataset.

Result: results of the study show that when the mechanism of missing reference data is known, accuracy measures can easily be correctly adjusted using either the Begg and Greenes method, or multiple imputation. In situations where the mechanism of missing reference data is complex or unknown, multiple imputation is more flexible and straight forward than the Begg and Greenes correction method.

Conclusion: partial verification by design can be a very efficient data collection strategy. In that case the pattern of missing reference data will be known and accuracy measures can easily be correctly adjusted using either Begg and Greenes method, or (Multiple) Imputation. If not defined by design, partial verification should be avoided, as it can seriously bias the results. There are however situations where the mechanism of missing reference data is not known and partial verification can not be avoided. In these situations we strongly recommend to use Multiple Imputation methods to correct. These methods are more flexible and straight forward than the Begg and Greenes correction method and give reliable estimates of the missing reference data.