OBJECTIVES: For randomized, interventional studies, the EUnetHTA 21 guideline on Validity of Clinical Studies recommends the Risk of Bias 2 (RoB 2) tool developed by the Cochrane group for assessing risk of bias as part of the Joint Clinical Assessment (JCA). In this study, we look at the extent to which the existing RoB 2 tool aligns with the principles of the ICH E9(R1) Addendum and identify relevant gaps.

METHODS: We critically review the Cochrane RoB 2 tool from the perspective of the ICH E9(R1) Addendum. We examine each signaling question and ask whether the question can be answered appropriately for a trial in which the endpoint was defined in terms of an estimand as described in ICH E9(R1). We identify areas where misalignment between the terminology used by the RoB 2 tool and ICH E9(R1) could cause confusion, leading to an inappropriate conclusion with respect to risk of bias or conflicting conclusions across different assessors.

RESULTS: The Cochrane RoB 2 tool does not easily accommodate assessment of endpoints in which the treatment effect of interest corresponds to hypothetical or while-on-treatment strategies for handling intercurrent events. For endpoints defined using a composite strategy, appropriate use of the RoB 2 depends on the subjective interpretation of the user, potentially resulting in conflicting conclusions among different assessors. Although the treatment policy and principal stratum strategies are most aligned with the two choices of treatment effects offered to users of the RoB 2 tool (i.e., ITT and per-protocol), nuances in the different concepts represented by ITT vs. treatment policy and per-protocol vs. principal stratum could still result in misapplication of the tool.

CONCLUSIONS: It is critical that tools for assessing risk of bias, such as RoB 2, are updated to reflect the relevant ICH guidelines that inform the design, conduct and interpretation of clinical studies.