EXPLICIT – DEVELOPING A PLATFORM FOR ELICITING EXPERT OPINION FOR MODEL-BASED ECONOMIC EVALUATIONS

Purpose:

   When evidence is scarce, experts are sometimes asked to estimate the uncertainty of model parameters as probability distributions. This process, usually referred to as expert elicitation, is complex and there is an ongoing debate on the best way to conduct elicitation. EXPLICIT (EXPert eLICItation Tool) was developed with three aims: 1) as an attempt to standardise the elicitation approach in accordance with established recommendations, 2) for the conduct of elicitation in the absence of a facilitator, and 3) as an environment for systematically exploring different approaches to elicitation.

Method(s):

   Starting from a number of reviews of elicitation methods and tools, an Excel-based tool was developed to assist the progress of the elicitation session, including the preparation of the expert and recording of the outcome.

   An early version included the histogram technique and a hybrid method to encode expert-elicited distributions as proportions; in later versions, a three-point estimation method (PERT) was also included for scalar quantities.

   Additional information on the elicitation task can be displayed in EXPLICIT, and consent can be recorded directly. Data on expert characteristics such as statistics literacy and years of experience can also be collected. Feedback from the participating experts has also been sought on the ease of use, as well as the ability of EXPLICIT to faithfully represent their beliefs.

   EXPLICIT was piloted on two different occasions, with clinical and non-clinical experts. In order to explore the feasibility of elicitation in the absence of a facilitator, some experts used EXPLICIT via email.

Result(s):

   The EXPLICIT tool has been used in four different elicitation tasks, involving 36 experts in total. 14 of them, completed the task unattended, via email; none of these experts had any experience with prior elicitation. Generally, experts found using EXPLICIT easy to use. Only one expert failed to use the tool and could not complete the elicitation task. There was wide agreement among experts that EXPLICIT could faithfully represent their uncertainty.

Conclusion(s):

   EXPLICIT was useful in both facilitating the elicitation task and in obtaining expert opinion from experts who could not be met face-to-face. Comparison of elicitation characteristics across studies was also possible. This work supports the opinion that, by using a standardised elicitation approach, more data could be aggregated in the methodological exploration of elicitation.