A Quantitative Assessment of the Economic Decision Drivers Outlined in HTA Outcomes Through Stratification and Sentiment Analysis: Evidence From Canada, England, Scotland, and Sweden

OBJECTIVES: Approaches to the economic evaluation of innovative medicines can vary significantly across HTA agencies. This research employs natural language processing techniques to contextualise the degree of severity of issues raised related to the economic evaluations in HTA.

METHODS: HTA data from Canada, England, Scotland, and Sweden were extracted from an internal HTA database (HTA-Hive), constructed using a validated and published framework for the assessment of HTA outcomes. Economic issues were categorised into: utilities, costs, modelling assumptions, the type of model selected, and sensitivity analyses conducted by the payer or the agency. A total of 497 HTA reports (spanning 2009-2024), with full economic assessments were included in the study. A sentiment analysis model VADER using the nltk package in Python was applied to the uncertainties to assess sentiment severity across agencies.

RESULTS: Out of a total of 1550 economic uncertainties, the most frequent related to modelling assumptions (32.3%), and the least frequent was model selection (1.9%). TLV had the worst average sentiment (0.06), significantly worse than NICE, INESSS (p<0.001). NICE was the most positive (0.40), significantly better than CADTH, SMC, and TLV (p<0.001). The average compound sentiment across all clinical uncertainties was 0.27. The category with the lowest average sentiment was model selection (0.19), highest was modelling assumptions (0.32). Sentiment scores were highly variable across manufacturers with more than 10 reports (0.068 – 0.512). Average sentiment was similar across orphan and non-orphan medicines (0.293 vs 0.269) and oncology drugs have a significantly higher average score than other drugs (0.371 vs 0.190).

CONCLUSIONS: More positive sentiment analysis for oncology drugs suggests that disease severity and unmet may positively influence the consideration of economic issues in HTA. Natural language processing techniques can help to contextualise the issues raised by agencies in their evaluation of economic evidence.