Presentation (CTI)

OBJECTIVES: To use 1000minds multi-criteria decision analysis tool to replicate a NICE Committee and compare recommendations from the software with real-world decision outcomes.
METHODS: We used the NICE website and NICE guidelines to determine the main factors that committees consider when making decisions on pharmaceutical and MedTech appraisals. We used the discrete choice event functionality of 1000minds software to apply weightings for clinical effectiveness, uncertainty, sustainability, equity, and health need. We combined those weightings with cost-effectiveness data from publicly available committee papers to see if an AI tool would make similar recommendations when compared with HTA (in this case NICE) decisions. Where there was discrepancy we went back and altered the weightings of the factors to see how much the impact varied or whether cost-effectiveness tended to drive recommendations. We used external review group (ERG) reports to summarise the key factors for each intervention.
RESULTS: The AI tool generally provided recommendations to our ‘HTA’ body consistently with existing NICE committees. However, when decisions did not match, it was unclear from publicly available reports why there was a mismatch. Varying our factor weights did not have substantial impact on the alignment of the recommendations. This indicates that committees are using human factors to consider each application and not necessarily applying consistent value to additional factors like clinical effectiveness, uncertainty, sustainability, equity, and health need.
CONCLUSIONS: It is often unclear what factors are important to HTA bodies when making recommendations. This is especially difficult when there are competing objectives such as maximising population health, reducing inequalities, operating withing a constrained, and reducing environmental impact. The current processes do not apply transparent weights to additional factors that are considered which can lead to inconsistent decision-making. A transparent process with appropriate consultation and public input to category selection and weights could help to focus committees and decision makers.