OBJECTIVES : Network meta-analysis (NMA) compares multiple health technologies when there is a lack of direct evidence with randomized controlled trials (RCT) alone. While supplementing RCTs with robust real-world evidence (RWE) in NMAs has become standard practice, NMAs of RWE alone are now being used to gain insights into the patient experience in clinical practice. This SLR aimed to assess the quality of published RWE-only NMAs and evaluate the statistical methods used to account for clinical and statistical heterogeneity.

METHODS : Comprehensive searches were conducted on PubMed and Embase to identify RWE-only NMAs, with no temporal or geographical limits. Study characteristics and statistical methodology were extracted for eligible studies following two-level screening. Quality of confounding adjustment and assessment of heterogeneity were evaluated.

RESULTS : Screening of 1,907 citations yielded five studies for inclusion in this SLR; all were published in the last two years. All NMAs were conducted per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Conditions where such analyses were conducted included cardiovascular (n=2), hepatological, hematological, and gastrointestinal (n=1 each) disorders. Only one NMA was industry-sponsored, while others were supported by academia or research grants. The NMAs used different risk-of-bias tools and assessed heterogeneity and inconsistency using appropriate tests depending on the nature of heterogeneity. Only one study conducted sensitivity analyses and two studies conducted multi-variable meta-regression to assess sources of heterogeneity. Methods accounting for heterogeneity were of mixed statistical rigor.

CONCLUSIONS : NMAs based on RWE alone are a relatively new and emerging methodological approach in comparative effectiveness. While the included studies assessed confounding bias, selection bias, and other sources of heterogeneity, questions remain about whether necessary adjustments were made to fully account for the degree of heterogeneity. Adequate statistical adjustments are critical if RWE-only NMAs are to be used to derive conclusions regarding real-world effectiveness of health technologies.