OBJECTIVES: Systematic literature review (SLR) guidance for best practice recommends the use of risk of bias (RoB) tools to determine the reliability of included studies. The aim of this research was to review which tools are most commonly used to assess bias in published SLRs.

METHODS: SLRs were identified via a pragmatic search using Ovid MEDLINE® 2021–2022 and selected for inclusion by a single reviewer if they exclusively included either randomised controlled trials (RCTs) or real-world evidence (RWE). Key data were extracted (disease area, study design and RoB tool).

RESULTS: SLRs of RCTs (n=45) most commonly used the Cochrane tools (ROB and ROB2). The original tool was used more frequently than the updated ROB2 tool (62.2% vs. 17.7%, respectively). The Jadad scale was the second most commonly used tool (13.3%).

SLRs of RWE studies (n=82) were more diverse in the selection of RoB tools. A total of 18 different tools were used, including the Newcastle–Ottawa scale (NOS), 46.3%; Joanna Briggs Institute (JBI) tools, 11.0%; ad hoc tools adapted from previous publications, 11.0%; and the ROBINS-I tool, 3.7%.

CONCLUSIONS: The Cochrane handbook is considered the gold standard for SLRs. Therefore the findings are not unexpected, in that the Cochrane tools were most commonly used in published SLRs of RCTs. However, the majority of SLR authors used the original tool and not the updated ROB2, which requires assessment per outcome. The reasoning for authors’ selection of the older tool should be investigated.

In SLRs of RWE studies there was a greater diversity in the tools used. The ROBINS-I tool, recommended by the Cochrane handbook, was used in just 3.7% of SLRs. The frequent use of ad hoc tools for RWE may reflect a greater difficulty in selecting tools appropriate for the range of study designs in this area.