Presentation (CTI)

OBJECTIVES: NICE HST (Highly Specialised Technologies) conducts appraisals of rare disease health technologies in UK. We tested a structured AI driven data extraction and analysis framework for evidence submitted for appraisal and derive key decision drivers for NICE recommendations in ultra-rare diseases.
METHODS: A Large Language Model (LLM) assisted platform was used to support synthesis and analysis of ultra-rare disease appraisal guidance documents published by NICE HST in 2023-24. The agent was trained using structured data extraction, reasoning, data-mapping and analysis framework and was prompted to search and evaluate evidence submitted and key decision drivers.
RESULTS: The AI driven approach yielded faster results than human approaches and was able to identify 10 guidance documents for unique assets/indications appraised by NICE HST and classify them on the basis of recommendations (recommended/not recommended, recommended only in research settings/specialised commissioning eg. Managed Access). 9 were recommended for use on the NHS and 1 was not recommended. Other details that were analysed were whether the recommendations covered the entire requested population or only restricted sub-groups and whether the ICER threshold was met/non met/borderline/undefined. Further, the evidence submitted and qualitative comments made by the committee with respect to each piece of evidence were evaluated. Key identified categories of evidence submitted included pivotal trials, cost-effectiveness modelling, resource impact modelling, generalisability to the NHS, natural history studies, models to extrapolate longer term/outcomes considered relevant by the NHS, current treatment patterns studies, time trade-off studies for carer disutility, registry studies, patient/carer surveys, case studies etc. We showcase effectiveness of AI to identify pieces of evidence that contributed as decision drivers in each NICE HST recommendation.
CONCLUSIONS: AI is a useful tool for synthesis and analysis of HTA decision drivers and assessing evidence evaluated in each appraisal. An AI based engine could potentially enhance and streamline the evidence planning and generation process process.