Navigating Uncertainty: Trends in ICER’s Evidence Ratings

OBJECTIVES: ICER’s evidence rating matrix assigns a letter grade to a therapy’s supporting evidence to capture the product's comparative net health benefit and its level of certainty. This study analyzed trends in ICER’s assigned grades over time across rare and non-rare diseases.

METHODS: Evidence rating grades and rare disease status for each assessed intervention (N=420) were collected from ICER's Final Evidence Reports for time frames corresponding to the past two frameworks (2017-2019, n=287; 2020-2023, n=133). Therapies were excluded if they were not pharmaceutical interventions, did not have completed Final Evidence Reports, did not have assigned letter grades, or did not have comparators. Therapy grades were categorized into ICER's collapsed evidence ratings (Uncertain, n=169; Comparable, n=128; Superior, n=123) and grouped based on rare vs non-rare disease (rare, n=85; non-rare, n=335).

RESULTS: For assessments conducted under the 2017-2019 framework, 37% of grades fell into the Uncertain category, 31% were Comparable, and 32% were Superior. For those under the 2020-2023 framework, the respective numbers were 47%, 29%, and 24%, representing a shift toward evidence uncertainty. For rare diseases across both time frames, the respective numbers were 49%, 15%, and 35%; for non-rare diseases, they were 38%, 34%, and 28%, representing greater uncertainty for rare diseases. The number of rare disease interventions graded during the 2017-2019 time frame (n=44) was comparable to the 2020-2023 time frame (n=41), but the number of non-rare disease interventions graded during 2017-2019 (n=243), was notably higher than the 2020-2023 time frame (n=92). The ratio of non-rare to rare disease interventions graded shifted from 5.9 in 2017-2019 to 2.1 in 2020-2023.

CONCLUSIONS: Over time, ICER has increasingly awarded Uncertain grades. This may be due to the relative shift toward assessments for interventions that treat rare diseases, to which ICER tends to assign Uncertain grades more frequently.