ARE BIOPHARMA DECISIONS BASED ON HIGH-QUALITY RWE? INSIGHTS FROM SLRS
Author(s)
Digant Gupta, MBBS, MPH1, Saifuddin Kharawala, MBBS, DPM1, Mudasir Khan, MPH1, Pankdeep Chhabra, MBBS, MPH, MSc1, Divyanshu Jindal, BCom, MBA1, Monique Martin, PharmD, MSc, MBA2, Paul Gandhi, MD, MBA, LLM2.
1Red Nucleus, Pune, India, 2Red Nucleus, London, United Kingdom.
1Red Nucleus, Pune, India, 2Red Nucleus, London, United Kingdom.
OBJECTIVES: Real-world evidence (RWE) increasingly informs HTA, regulatory, payer, and biopharmaceutical decision-making. However, the methodological quality of observational studies underpinning these decisions has not been systematically characterized. We evaluated the quality of studies included in RWE SLRs and identified factors associated with high-quality.
METHODS: An internal database of 52 SLRs spanning 45 indications and 14 therapeutic areas was used as the basis for this analysis, yielding 7,351 RWE studies. Study quality was assessed via the Newcastle-Ottawa Scale (NOS; low:0-5, moderate:6-7, high:8-9) using OpenAI's GPT 5.4 mini model. Multivariable logistic regression identified study characteristics independently associated with high-quality.
RESULTS: Overall, 31.1%, 40.6%, and 28.3% studies were classified as low, moderate, and high-quality, respectively, with a mean NOS score of 6.3 (median 6.0; range: 4.3─7.5). Study design was the strongest predictor, with cohort studies six times more likely to be high-quality than cross-sectional studies (adjusted odds ratio [aOR]=5.99; 95% CI: 5.10-7.03). Sample size was the next strongest predictor, with studies in the largest quintile nearly nine times more likely to be high-quality than those in the smallest (aOR=8.95; 95% CI: 7.15-11.22). Therapeutic area was also associated with study quality, with nephrology (aOR=7.51), oncology (aOR=6.59), and rheumatology (aOR=6.44) demonstrating the highest odds of high-quality relative to immunology. Funding source was independently associated with study quality; non-industry-funded studies had higher odds of high-quality than studies with undisclosed funding (aOR=1.46; 95% CI: 1.24-1.72), whereas no significant difference was observed for industry-funded studies.
CONCLUSIONS: The methodological quality of RWE studies included in SLRs was highly variable, with fewer than one-third meeting high-quality criteria. Study design and sample size emerged as the strongest determinants of methodological quality. These findings provide a large-scale benchmark of RWE quality and identify study characteristics associated with higher-quality evidence, informing evidence generation and healthcare decision-making.
METHODS: An internal database of 52 SLRs spanning 45 indications and 14 therapeutic areas was used as the basis for this analysis, yielding 7,351 RWE studies. Study quality was assessed via the Newcastle-Ottawa Scale (NOS; low:0-5, moderate:6-7, high:8-9) using OpenAI's GPT 5.4 mini model. Multivariable logistic regression identified study characteristics independently associated with high-quality.
RESULTS: Overall, 31.1%, 40.6%, and 28.3% studies were classified as low, moderate, and high-quality, respectively, with a mean NOS score of 6.3 (median 6.0; range: 4.3─7.5). Study design was the strongest predictor, with cohort studies six times more likely to be high-quality than cross-sectional studies (adjusted odds ratio [aOR]=5.99; 95% CI: 5.10-7.03). Sample size was the next strongest predictor, with studies in the largest quintile nearly nine times more likely to be high-quality than those in the smallest (aOR=8.95; 95% CI: 7.15-11.22). Therapeutic area was also associated with study quality, with nephrology (aOR=7.51), oncology (aOR=6.59), and rheumatology (aOR=6.44) demonstrating the highest odds of high-quality relative to immunology. Funding source was independently associated with study quality; non-industry-funded studies had higher odds of high-quality than studies with undisclosed funding (aOR=1.46; 95% CI: 1.24-1.72), whereas no significant difference was observed for industry-funded studies.
CONCLUSIONS: The methodological quality of RWE studies included in SLRs was highly variable, with fewer than one-third meeting high-quality criteria. Study design and sample size emerged as the strongest determinants of methodological quality. These findings provide a large-scale benchmark of RWE quality and identify study characteristics associated with higher-quality evidence, informing evidence generation and healthcare decision-making.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR293
Topic
Health Technology Assessment, Methodological & Statistical Research, Real World Data & Information Systems
Disease
No Additional Disease & Conditions/Specialized Treatment Areas