OBJECTIVES: The Highly Specialized Technologies (HST) process, employed by the National Institute for Health and Care Excellence (NICE) in the UK, evaluates technologies for severe rare diseases. Randomized-controlled trials (RCT) may be infeasible for rare diseases, thus making real-world evidence (RWE) a valuable source of information. Acceptance of RWE has been rising in Health Technology Assessment (HTA) as demonstrated by the introduction of the NICE RWE framework in June 2022. We aimed to determine how RWE is used within HST submissions following the introduction of the NICE RWE framework.

METHODS: A targeted review of all NICE HST submissions published between June 2022 and May 2024 was conducted. Relevant information from company submissions, evidence assessment group (EAG) reports, and committee appraisals was extracted. We then explored how RWE was utilized by the company and how it was perceived by the EAG and NICE committee.

RESULTS: Ten HST submissions were assessed, nine of which received a positive recommendation. RWE was included in 9/10 HSTs, to assess generalizability of analysis findings, estimate effects of interventions, or support trial evidence. The most commonly used real-world data sources included patient registries, observational studies and health surveys.

The EAG and committee accepted the use of RWE in most HSTs, but frequently noted concerns regarding quality assessment, internal validity of studies and uncertainty of findings. To overcome uncertainties with clinical and cost-effectiveness, the EAG recommended collection of additional long-term real-world data in 5/10 HSTs. In one HST, the committee specifically recommended usage of the RWE framework to improve the robustness of the clinical evidence.

CONCLUSIONS: Most NICE HST submissions used RWE to validate trial/analysis findings in the general population. Despite its acceptance by the EAG and the committee, manufacturers should be aware that high uncertainty and insufficient quality assessment of RWE can lead to criticism.