Presentation (CTI)

OBJECTIVES: The parameters included in economic evaluations are often uncertain. Structured expert elicitation (SEE) refers to a systematic protocol for extracting expert judgements about an unknown quantity and formulating this information as a continuous probability distribution. In 2022, the National Institute for Health and Care Excellence (NICE) recommended using structured protocols to minimise bias and uncertainty when eliciting expert judgements. The objective is to explore whether (and how) SEE was used in NICE submissions between 2022 and 2025.
METHODS: Technical appraisal (TA) and highly specialist technology (HST) submissions referring to “expert elicitation” were examined. Information concerning how the SEE protocols were used, quantities of interest and external assessment group (EAG) comments were extracted. HST submissions were further examined to assess the frequency of unstructured elicitation approaches (e.g. interviews).
RESULTS: Overall, 15 guidelines were identified that referred to expert elicitation. Five submissions (HST and TA) used SEE to inform input parameters, including clinical outcomes for standard of care and expected survival. The MRC, SHELF and IDEA protocols and modified Delphi approaches were referenced. Where SEE was used, elements of the decision problem were detailed, such as the methods used for judgement encoding, distribution fitting and aggregation. Five HST submissions employed unstructured expert elicitation to source inputs. One submission noted that SEE was infeasible due to disease rarity and the consequent lack of available expertise. There were five submissions that did not use elicitation. Here, the EAG recommended eliciting expert judgements to address evidence paucity, which could significantly influence value estimates associated with the new treatments.
CONCLUSIONS: Our review found that expert elicitation has informed recent NICE submissions; however, elicitation was often unstructured. Furthermore, inconsistencies among protocols (e.g. aggregation approach) limited comparability between submissions.