PURPOSE: To discuss best practices, challenges, and the increasing use of Structured Expert Elicitation (SEE) in the context of healthcare decision-making and demonstrate its applicability through an interactive, live experiment.

DESCRIPTION: In a landscape of accelerated approvals, a less mature evidence base, and increasing reliance on real world evidence, obtaining judgements from clinical experts is becoming increasingly important to inform healthcare decision-making (HCDM). In the past, expert elicitation has generally been unstructured in nature, but growing demand has led to a desire to improve the robustness of the methods used.

In November 2020, NICE released a report on the case for change of its methods for health technology assessments (HTAs). One topic discussed here is the use of expert elicitation in a structured, quantitative manner. It is expected that this form of elicitation – particularly the elicitation of uncertain quantities – will be increasingly used to inform submissions to NICE. In light of this anticipated change, we will provide an interactive demonstration of how SEE may be used in practice to inform HTA, and discuss best practices in SEE, challenges in its implementation in HCDM based on our recent experiences, and its expected use in the future.

James Horscroft will summarize the different approaches to SEE and set up a live SEE experiment, using the audience as the experts. Jeremy Oakley will present an overview of the Sheffield Elicitation Framework method, how it should be conducted, its advantages and disadvantages, and its applicability to HCDM. Laura Bojke will discuss her recent work on the use of SEE in an HTA context, the dos and don’ts, and what may constitute a set of minimum standards for SEE applied to HCDM. Finally, the panel will review and discuss the results of the experiment and provide their thoughts on the future of SEE in HTA.