OBJECTIVES: Over the years, advanced methods such as network meta-analysis (NMA) have become integral in the realm of evidence synthesis, which pools data from different sources and allows for both direct and indirect evidence. Nevertheless, conducting an NMA poses a significant challenge due to the inherent heterogeneity between the studies. This study aims to understand (1) the causes of heterogeneity in published literature, and (2) how it has been handled in single technology appraisals (STAs) of NICE in recent years.

METHODS: Desk research was performed to identify the sources of heterogeneity in NMA. For the second objective, we reviewed the final guidance of STAs published by the NICE in the last 2 years (January 2022–November 2023) for non-oncology indications. Terminated, withdrawn, and in-development STAs were excluded.

RESULTS: A quick glance of the literature identified three main types of heterogeneity, including clinical (i.e., differences in patient characteristics, treatments, outcomes), methods-related (i.e., study design/conduct), and statistical (i.e., chance). Nevertheless, the root cause of heterogeneity in studies was the existence of interaction between treatment effect and study-level covariates, i.e., presence of treatment effect modifiers. Failure to properly address the heterogeneity led to biased results.

CONCLUSIONS: The study shows that heterogeneity in NMA has been addressed poorly in non-oncology studies. Despite the availability of various guidelines/task-force recommendations on conducting NMA, the adherence to these guidelines for addressing heterogeneity is limited.