Utilizing Expert Consensus Through Delphi Panels in Reimbursement Decisions in England: A Retrospective Analysis of NICE Appraisals

OBJECTIVES: The National Institute for Health and Care Excellence (NICE) recognizes three formal consensus methods (Delphi panel, nominal group technique, and consensus development conference) to address uncertainties and evidence gaps when appraising health technologies. To date, only Delphi panels have been utilized to reach expert consensus by submitting companies. This study aimed to explore the extent to which NICE accepts the use of Delphi panels to inform its appraisals of health technologies in England, and how this evidence is perceived and incorporated into decision-making.

METHODS: A retrospective analysis was conducted on NICE appraisals published between 2016 and 2023 that utilized Delphi panels. Information was extracted on the disease area, treatment line, rarity of the condition, number of panelists, domains addressed by the Delphi panel, and the NICE committee's perception and utilization of this evidence.

RESULTS: Eleven NICE appraisals were identified, with 64% (7/11) in rare diseases. The Delphi panels consisted of 4–20 clinical experts considering the five appraisals where the number of panelists was disclosed, and explored various domains, including resource use estimates, disease progression rates, treatment persistence, utility values, morbidity, and mortality. Information on number of rounds conducted until consensus across domains was generally not disclosed in redacted documents. NICE acknowledged the limitations of expert opinion while also recognizing its value in addressing specific knowledge gaps, particularly for rare diseases or long-term outcomes.

CONCLUSIONS: As a recognized consensus method by NICE, Delphi panels provided valuable expert opinion evidence that was used judiciously to support health technologies’ data package, particularly when outcomes could not be demonstrated within trial or real-world settings. While acknowledging the limitations of expert opinion in general, NICE committees incorporated the evidence from Delphi panels to inform their decision-making, especially for rare diseases or long-term outcomes where robust clinical data may be limited.