Comparing ROBINS-I V2 and Downs Black for nRCTs in the Context of HTA
Author(s)
RANJITH BABU VAGICHARLA, M.Pharm1, Aditi Aggarwal, M.Pharm1, Manne Mithun Chakrawarthy, M.Pharm2, Geddamuri Bala Ganesh, M.Pharm1, Premsagar Kummari, M.Pharm1, Veeranki Prashanth, M.Pharm2, Yogesh Punekar, .3.
1IQVIA, Bengaluru, India, 2IQVIA, Gurgaon, India, 3Principal-RWE, IQVIA, London, United Kingdom.
1IQVIA, Bengaluru, India, 2IQVIA, Gurgaon, India, 3Principal-RWE, IQVIA, London, United Kingdom.
OBJECTIVES: Health technology assessment (HTA) bodies are increasingly recognising the value of non-randomised clinical trials (nRCTs). These studies are employed when randomised clinical trial (RCT) data is limited or unavailable, as nRCTs are positioned just below RCTs in the PRISMA evidence hierarchy. Risk of bias in non-randomised studies - of interventions (ROBINS-I) and the Downs & Black checklists are widely used critical appraisal tools for evaluating nRCTs. The Downs & Black tool, published in 1998, is valued for its simplicity and time efficiency. In contrast, the updated ROBINS-I Version 2 (V2), released in November 2024, offers enhanced detail, refined signalling questions, and improved judgement logic. This review compares ROBINS-I V2 and Downs & Black using published nRCTs, focusing on domain-level outputs and complexity in the context of past HTA recommendations.
METHODS: Three published nRCTs in endometrial cancer were assessed using both tools. Domain-level judgements (evaluated across 10 domains), scoring styles, and usability were compared. HTA methods guidelines from NICE, SMC, NCPE, HAS, and G-BA were reviewed to understand current study assessment practices.
RESULTS: ROBINS-I V2 evaluates seven domains, including confounding, selection, intervention classification, deviations, missing data, outcome measurement, and selective reporting. It employs qualitatively structured, algorithm-guided signalling questions with refined response options, requiring expert judgement. In contrast, Downs & Black is a checklist-based tool that quantitatively assesses reporting, external/internal validity, and statistical power, but lacks detailed scoring guidance. In the studies evaluated, moderate to serious bias particularly from confounding and selective reporting was observed using ROBINS-I V2. The same studies scored favourably (18-24/27) using the Downs & Black tool, highlighting broader reporting and external validity but offering limited bias-specific depth.
CONCLUSIONS: ROBINS-I V2 provides a granular, bias-oriented evaluation, whereas the Downs & Black tool is quicker to apply but less comprehensive per domain. ROBINS-I V2 offers a more thorough and HTA-aligned assessment.
METHODS: Three published nRCTs in endometrial cancer were assessed using both tools. Domain-level judgements (evaluated across 10 domains), scoring styles, and usability were compared. HTA methods guidelines from NICE, SMC, NCPE, HAS, and G-BA were reviewed to understand current study assessment practices.
RESULTS: ROBINS-I V2 evaluates seven domains, including confounding, selection, intervention classification, deviations, missing data, outcome measurement, and selective reporting. It employs qualitatively structured, algorithm-guided signalling questions with refined response options, requiring expert judgement. In contrast, Downs & Black is a checklist-based tool that quantitatively assesses reporting, external/internal validity, and statistical power, but lacks detailed scoring guidance. In the studies evaluated, moderate to serious bias particularly from confounding and selective reporting was observed using ROBINS-I V2. The same studies scored favourably (18-24/27) using the Downs & Black tool, highlighting broader reporting and external validity but offering limited bias-specific depth.
CONCLUSIONS: ROBINS-I V2 provides a granular, bias-oriented evaluation, whereas the Downs & Black tool is quicker to apply but less comprehensive per domain. ROBINS-I V2 offers a more thorough and HTA-aligned assessment.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
HTA84
Topic
Health Technology Assessment, Methodological & Statistical Research
Disease
No Additional Disease & Conditions/Specialized Treatment Areas