Exploring the Evaluation of Network Meta-Analysis Assumptions: Current Approaches and Practices
Author(s)
Kansak Boonpattharatthiti, PharmD1, Kanyaphak Chueadi, PharmD1, Phiyanuch Thimkorn, PharmD1, Deborah M Caldwell, Ph.D.2, Nathorn Chaiyakunapruk, PharmD, PhD3, Teerapon Dhippayom, PharmD, PhD1.
1The Research Unit of Evidence Synthesis (TRUES), Faculty of Pharmaceutical Sciences, Naresuan University, Phitsanulok, Thailand, 2Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom, 3Department of Pharmacotherapy, College of Pharmacy, University of Utah, Salt Lake City, UT, USA.
1The Research Unit of Evidence Synthesis (TRUES), Faculty of Pharmaceutical Sciences, Naresuan University, Phitsanulok, Thailand, 2Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom, 3Department of Pharmacotherapy, College of Pharmacy, University of Utah, Salt Lake City, UT, USA.
OBJECTIVES: Network meta-analysis (NMA) enables the comparison of multiple treatments simultaneously by combining direct and indirect evidence. Its use in medical decision-making has increased rapidly. However, the validity of NMA results relies on key assumptions, i.e. homogeneity, transitivity, and consistency. If these assumptions are not consistently evaluated, the reliability of the findings may be affected. This study aims to explore how researchers assess these assumptions in published NMA studies.
METHODS: A cross-sectional study was conducted on published NMA articles. We searched PubMed, Embase, and Cochrane CENTRAL from January 2010 to August 2024. A total of 22,079 articles were identified. We calculated the sample size from this known population and used systematic random sampling to select 393 NMAs using a random number table. Each NMA was independently extracted by two of four research assistants, focusing on NMA assumptions, which were analyzed descriptively.
RESULTS: Of the 393 NMAs, 282 NMAs (71.8%) were published between 2020 and 2024. A total of 37.7% reported adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for NMA (PRISMA-NMA). Most NMAs (n=362) used a randomized controlled trial study design. Oncological, cardiovascular, and neurological disorders were the top three disease conditions studied. Homogeneity was assessed in 300 NMAs (76.3%), transitivity in 45 NMAs (11.5%), and consistency in 265 NMAs (67.4%). Only 38 NMAs (9.7%) assessed all three assumptions. Among 282 NMAs published between 2020 and 2024, 210 NMAs (74.5%) assessed the homogeneity assumption, 35 NMAs (12.4%) assessed the transitivity assumption, and 207 NMAs (73.4%) assessed the consistency assumption. NMAs published in high-impact journals more frequently evaluated these assumptions compared to those in low-impact journals.
CONCLUSIONS: The assessment of NMA assumptions, especially transitivity and consistency, varied widely across studies. These findings emphasize the importance of standardized protocols or reporting guidelines to ensure proper and transparent assessment.
METHODS: A cross-sectional study was conducted on published NMA articles. We searched PubMed, Embase, and Cochrane CENTRAL from January 2010 to August 2024. A total of 22,079 articles were identified. We calculated the sample size from this known population and used systematic random sampling to select 393 NMAs using a random number table. Each NMA was independently extracted by two of four research assistants, focusing on NMA assumptions, which were analyzed descriptively.
RESULTS: Of the 393 NMAs, 282 NMAs (71.8%) were published between 2020 and 2024. A total of 37.7% reported adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for NMA (PRISMA-NMA). Most NMAs (n=362) used a randomized controlled trial study design. Oncological, cardiovascular, and neurological disorders were the top three disease conditions studied. Homogeneity was assessed in 300 NMAs (76.3%), transitivity in 45 NMAs (11.5%), and consistency in 265 NMAs (67.4%). Only 38 NMAs (9.7%) assessed all three assumptions. Among 282 NMAs published between 2020 and 2024, 210 NMAs (74.5%) assessed the homogeneity assumption, 35 NMAs (12.4%) assessed the transitivity assumption, and 207 NMAs (73.4%) assessed the consistency assumption. NMAs published in high-impact journals more frequently evaluated these assumptions compared to those in low-impact journals.
CONCLUSIONS: The assessment of NMA assumptions, especially transitivity and consistency, varied widely across studies. These findings emphasize the importance of standardized protocols or reporting guidelines to ensure proper and transparent assessment.
Conference/Value in Health Info
2025-09, ISPOR Real-World Evidence Summit 2025, Tokyo, Japan
Value in Health Regional, Volume 49S (September 2025)
Code
RWD311
Topic Subcategory
Reproducibility & Replicability
Disease
No Additional Disease & Conditions/Specialized Treatment Areas