OBJECTIVES: To develop a guide for non-statisticians when reviewing published network meta-analyses (NMAs). We aim at summarising key concepts behind NMAs and the implications for external reviewers.

METHODS: Recently published NMAs were reviewed. A workshop was organised with non-statisticians to identify the key concepts and outputs from NMAs leading to confusion on the conclusions and/or implications. Comprehensive definitions were formulated along with the implications of published NMAs’ characteristics.

RESULTS: Non-statisticians found that published NMAs tend to present the key methodological choices made without clearly stating the associated limitations, and noted that some graphical presentations of results could directly impact the interpretation of the magnitude of potential differences. Some key methodological choices include the use of Bayesian over frequentist NMAs or the use of random-effect vs. fixed-effect models. To respond to the challenges identified while reviewing published NMAs, this guide included key definitions related to the methods and outputs of NMAs, such as SUCRA, the strengths and limitations related to certain methodological choices, the interpretation and potential shortcomings of graphical displays.

CONCLUSIONS: This guide has been designed to help non-statisticians critically review published NMAs including with regards to methodological choices and types of outputs used.