Methodology and Challenges of Network Meta-Analysis in Health Technology Assessment of Medical Devices: A Case Study of Drug-Eluting Stents
Author(s)
Huey Yi Chong, PhD1, Rebecca Hughes, BSc2, Ayesha Rahim, MSc1, Megan Dale, MSc1.
1Cardiff and Vale UHB, Cardiff, United Kingdom, 2CEDAR, Cardiff, United Kingdom.
1Cardiff and Vale UHB, Cardiff, United Kingdom, 2CEDAR, Cardiff, United Kingdom.
OBJECTIVES: In the National Institute of Health and Care Excellence (NICE) late-stage assessment of drug eluting stents (DES), a Bayesian network meta-analysis (NMA) was undertaken to estimate the relative treatment effect between stents. The NMA compared 10 devices using 14 randomised controlled trials (RCTs). Data sparsity and the lack of prior information posed significant challenges in generating precise results. We aim to report and compare the methodology and challenges of NMA stents.
METHODS: Using the NICE DES assessment targeted literature searches of 8 bibliographic databases, any NMAs of DES in any populations were included. Data including NMA method, model specification, sensitivity analyses, results and key limitations were extracted.
RESULTS: A total of 3 NMAs were identified. Bayesian framework was employed in two studies, and one used Frequentist approach. None of the studies reported information on model fitting. Two NMAs included 77-147 RCTs with 99,039-126,526 participants, while one NMA included 4 studies with 6,637 participants with high bleeding risk. The NMAs compared 6-12 devices including bare metal stent. Wide 95% credible intervals (CrIs) were reported for some pairwise comparisons in the 2 Bayesian models, particularly when comparing the devices with the largest and smallest sample sizes. In the Frequentist NMA, 95% confidence intervals were narrow for all pairwise comparisons. Sensitivity analyses were performed in all NMA studies. In the sensitivity analysis using sparser data when the number of studies was reduced, NMA results became unstable, or produced even wider 95% CrIs.
CONCLUSIONS: Conducting NMA in medical devices is challenging when data are sparse, leading to uncertainty with the estimates. While RCTs are gold standard for evaluating efficacy, the incentive for conducting RCTs in medical devices is limited given their rapidly evolving nature. Generating comparative real-world data is also challenging, however it may offer an alternative to bridge this evidence gap.
METHODS: Using the NICE DES assessment targeted literature searches of 8 bibliographic databases, any NMAs of DES in any populations were included. Data including NMA method, model specification, sensitivity analyses, results and key limitations were extracted.
RESULTS: A total of 3 NMAs were identified. Bayesian framework was employed in two studies, and one used Frequentist approach. None of the studies reported information on model fitting. Two NMAs included 77-147 RCTs with 99,039-126,526 participants, while one NMA included 4 studies with 6,637 participants with high bleeding risk. The NMAs compared 6-12 devices including bare metal stent. Wide 95% credible intervals (CrIs) were reported for some pairwise comparisons in the 2 Bayesian models, particularly when comparing the devices with the largest and smallest sample sizes. In the Frequentist NMA, 95% confidence intervals were narrow for all pairwise comparisons. Sensitivity analyses were performed in all NMA studies. In the sensitivity analysis using sparser data when the number of studies was reduced, NMA results became unstable, or produced even wider 95% CrIs.
CONCLUSIONS: Conducting NMA in medical devices is challenging when data are sparse, leading to uncertainty with the estimates. While RCTs are gold standard for evaluating efficacy, the incentive for conducting RCTs in medical devices is limited given their rapidly evolving nature. Generating comparative real-world data is also challenging, however it may offer an alternative to bridge this evidence gap.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
SA68
Topic
Clinical Outcomes, Medical Technologies, Study Approaches
Topic Subcategory
Meta-Analysis & Indirect Comparisons
Disease
Cardiovascular Disorders (including MI, Stroke, Circulatory), Surgery