OBJECTIVES

Results of expert elicitation (EE) exercises can provide significant input to economic evaluations submitted to the National Institute for Health and Care Excellence (NICE) Single Technology Appraisal (STA) process in cases where empirical evidence is lacking or poor. However, the robustness of approaches to EE has the potential to vary considerably. This analysis aimed to evaluate the current use, reporting, scrutiny and acceptance of EE in NICE STAs.

METHODS

STAs published between October 2018 and April 2019 were reviewed. Where ‘inputs’ (model inputs or assumptions) for the manufacturer’s economic model had been informed or validated by experts, characteristics related to that input were extracted; such characteristics included additional justification from the manufacturer to support the use of the EE output, whether the input was included in scenario analyses, and Evidence Review Group (ERG) criticism.

RESULTS

Twenty-five STAs were included, of which only two submissions did not report any use of EE. EE was used for 173 inputs, an average of 6.9 inputs per submission. The most common methods of elicitation were ad-hoc validation (55.1%) and advisory boards (38.8%). Quality of reporting of EE methods varied substantially. 60.1% of inputs were unsupported by additional justification and 67.1% were not varied in scenario analyses; of 73 inputs that were unsupported by additional justification and not varied in scenarios analyses, the ERG only commented on 16.4%. Of 19 inputs that were unsupported by additional justification and shown to impact on cost-effectiveness in scenario analyses (incremental cost-effectiveness ratio change >1%), the ERG commented on 42.1%.

CONCLUSIONS

Although EE is consistently used in company submissions for NICE STAs, the level of reporting varies substantially. The frequent lack of justification, exploration of alternative scenarios and critique by the ERG may imply a degree of assumed trust and acceptance of expert opinion, even when an input may impact upon cost-effectiveness.