Objective: Reimbursement decision-making for rare disease treatments can be complex. Under many frameworks, convincing (high quality) evidence leads to strong reimbursement recommendations, while unconvincing (low quality) evidence leads to weaker recommendations; however, this may disadvantage access to rare disease treatments with lower amounts of evidence overall, and clinical heterogeneity within small populations. The objective was to evaluate the appropriateness of commonly-used value tools for assessing clinical evidence for rare disease treatments.

Methods: A targeted review of published literature and payer websites was conducted to identify assessment frameworks frequently used to evaluate evidence for rare disease treatments. Available criteria were assessed against common features of conducting research in rare diseases.

Results: Identified frameworks included the Grading of Recommendations Assessment, Development and Evaluation (GRADE), an adaptation of the Population/Intervention/Comparators/Outcomes/Study design (PICOS) criteria to an evaluation framework, and the Clinical evidence of Orphan Medicinal Products Assessment (COMPASS) tool (a rare-disease-specific tool used by the European Medicines Agency). These frameworks include assessment criteria that evaluate features known to be challenges in rare disease research such as small populations, small number of studies, and significant heterogeneity. Consequently, non-rare-disease-specific frameworks would result in clinical evidence from rare disease trials having reduced quality scores, regardless of the feasibility or ethical considerations of conducting larger randomized controlled studies.

Conclusions: Non-rare-disease-specific assessment frameworks are not fully-suited to evaluating evidence for new rare disease treatments due to characteristics of data available for these treatments. Using tailored criteria specific to rare diseases would allow for a more nuanced approach to assessing evidence quality; and may lead to greater consistency between assessments, improved differentiation between products, and more timely access to effective treatments for patients with rare diseases.