OBJECTIVES: The Institute for Clinical and Economic Review’s (ICER’s) Evidence Rating Matrix is a comparative clinical effectiveness decision tool that distills an intervention’s net health benefit and its uncertainty into ordered categorical grades. ICER assessments also estimate incremental lifetime quality-adjusted life years (QALYs) through cost-effectiveness modeling. This analysis aims to explore the relationship between ICER’s Evidence Rating Matrix and incremental QALYs.

METHODS: We used ICER’s Evidence Compendium and corresponding reports to compile a list of pairwise treatment comparisons, their respective Evidence Ratings, and incremental QALY findings. We limited the analysis to consistent pairwise comparisons that used a lifetime time horizon and were published between January 2017 and September 2021. For each pairwise comparison, we collapsed Evidence Ratings into three categories based on estimated net-health benefit: Superior (including A, B, B+), Comparable or Better (including C+, C++), and Uncertain (including P/I, I). Descriptive outcomes by collapsed category included mean incremental QALYs and the proportion with incremental QALYs ≥ 0.5.

RESULTS: Eighty-three pairwise comparisons were included. Mean incremental QALYs associated with each collapsed rating category were: Superior 2.50 (N = 65), Comparable or Better 0.95 (N = 7), and Uncertain 0.40 (N = 11). The proportion of comparisons with ≥0.5 incremental QALYs by collapsed rating category were: Superior 84.6%, Comparable or Better 71.4%, and Uncertain 18.2%.

CONCLUSIONS: We observed that better comparative clinical effectiveness ratings were associated with larger health gains when estimated using lifetime cost-effectiveness models. Although unique, ICER’s Evidence Ratings and incremental QALYs contain overlap within their goals. This overlap is supported by the findings from this descriptive study and may be useful in maintaining consistency across assessments.