Assessing the Effect of Risk of Bias Assessments in Randomized Controlled Trials of Pharmacological Interventions with Different Types of Outcomes and Comparators – a Bayesian Meta-Epidemiological Study

OBJECTIVES: Methodological faults in randomized controlled trials (RCTs) may introduce bias in effect estimates and lead to incorrect conclusions and, consequently, inappropriate health policy decisions. The effects of potential biases across RCTs of pharmacological interventions is yet unknown. We sought to investigate if differences exist in the effects of risk of bias (RoB) assessments on results from pharmacological RCTs when comparing studies with subjective and objective outcomes and with active and inactive comparators.

METHODS: We used a dataset of 1738 RCTs from 157 Cochrane meta-analyses and fitted Bayesian meta-epidemiological models to estimate associations between RoB (for sequence generation, allocation concealment, blinding, and incomplete outcome data) and RCT effect sizes. Outcome measures were classified as objective or subjective and comparators as active or inactive. Results were reported as ratio of odds ratios (ROR) with 95% credible intervals (CrI).

RESULTS: Effect sizes from inactive comparators were, on average, overestimated in trials with high or unclear RoB (versus low), for sequence generation and allocation concealment, regardless of the type of outcome. For the blinding domain, we only found a statistically significant overestimation for objective outcomes, despite a tendency in the same direction has also been observed for subjective outcomes. Conversely, for active comparators we did not detect any differences in effect sizes between trials with high or unclear RoB (versus low), either for objective or subjective outcomes. Bias related to incomplete outcome data did not show to influence effect sizes in any outcome-comparator pair type.

CONCLUSIONS: Our study suggests that in pharmacological RCTs, results from head-to-head trials are not modified by RoB assessments, whereas effect sizes from trials with inactive comparators tend to be overestimated when high or unclear RoB is present. Consequently, regardless of the degree of outcome subjectivity, potentially biased placebo-controlled trials might be misleading when informing decisions about approval and coverage of health technologies.