COMPOSITE EVALUATIONS OF TESTS: MULTIPLE PRIMARY STUDIES INTEGRATED WITH AN ECONOMIC MODEL. EXPERIENCE WITH TWO CASES

Purpose: To inform policy makers, a “linked evidence” approach may be used to evaluate the effectiveness and cost-effectiveness of diagnostic tests. This can involve conducting multiple primary studies on the accuracy and impacts of diagnostic testing which are then linked via an economic model. We present our experience of the challenges conducting such studies alongside development of economic models. We refer to this type of research, where multiple primary studies are integrated with an economic model, as the composite evaluation of tests.

Method(s): We have experience of composite evaluation in two completed projects:  the effectiveness and cost-effectiveness of (i) strategies to maximise identification of single gene diabetes (MODY), (ii) school entry hearing screening. In a case study approach we have reflected on the strengths and weaknesses of composite evaluation, particularly challenges encountered in both projects.

Result(s):

Both studies provided evidence on effectiveness and cost-effectiveness. They did this in a time that was less than might be taken for an RCT (approximately 4-5 years). However, due to the rarity of MODY and hearing problems at school entry in the UK, it is unlikely that an RCT would have been feasible in either case. Whether the research findings were as credible as expected from RCTs awaits the verdict of readers of the final publications (MODY paper submitted, hearing paper in press). We identified many challenges and will focus on:

• Complexity of the approach

• Clearly establishing areas of greatest uncertainty when designing the primary studies

• Tendency to over-elaborate the model

• Justifying use of data from the primary studies over data/studies which might already exist

• Importance of clear communication between researchers conducting the primary studies and those developing and analysing the model

• Difficulty of publishing all clinical studies, resulting in loss of transparency in the model

• Challenge of identifying this approach as being distinct from economic modelling without new data collection

The presentation will expand on the above areas.

Conclusion(s): Composite evaluation of tests is a useful additional method complementing experimental and quasi-experimental primary studies to evaluate test impact on patient outcome. They are not however a panacea and require resources and time to do well. The challenges we have identified will hopefully aid others preparing similar approaches in the future.