Author(s)
Hong YD1, Orsini L2, Guerino J2, Berger M3, Betz GD4, Crown W5, Goettsch W6, Mullins CD1, Willke R2, Jansen JP7
1University of Maryland School of Pharmacy, Baltimore, MD, USA, 2ISPOR, Lawrenceville, NJ, USA, 3Independent Consultant, New York, NY, USA, 4Health Sciences and Human Services Library, University of Maryland, Baltimore, MD, USA, 5The Heller School for Social Policy and Management, Brandeis University, Waltham, MA, USA, 6Utrecht Centre of Pharmaceutical Policy, Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht University, Utrecht, Netherlands; National Health Care Institute, Diemen, Netherlands, 7Department of Clinical Pharmacy, School of Pharmacy, University of California – San Francisco, USA; PrecisionHEOR, Oakland, CA, USA
OBJECTIVES: There have been ongoing efforts to determine whether data from observational studies can be applied to clinical and regulatory decision making. The objective of this systematic review of reviews was to determine how relative treatment effects of pharmaceuticals differ between observational studies and randomized controlled trials (RCTs). METHODS: We searched PubMed and Embase for systematic reviews published between January 1990 and January 2020. The search focused on comparative assessments of pharmaceuticals in the English language. We included reviews designed to compare relative treatment effects from observational studies with the corresponding effects from RCTs, and reviews that conducted subgroup analyses by study design. We extracted pooled relative effect estimates of RCTs and observational studies for each outcome, intervention-comparator, or indication assessed in the reviews. A ratio of ratios (RoR) was calculated comparing effects from RCTs relative to effects from observational studies, and we evaluated the extent of the difference between the relative treatment effect estimates. RESULTS: Thirty reviews spanning 7 therapeutic areas (cardiovascular disease [15/30], infectious disease [6/30], oncology [3/30], mental health [2/30], immune-inflammatory [1/30], metabolic disease [1/30], and other [2/30]) met our inclusion criteria. The included reviews analyzed data from 519 studies. Seventy-four relative effect estimate pairs (hazard ratios, risk ratios or odds ratios) from 29 reviews that reported pooled relative effect estimates for RCTs and observational studies, including a variety of outcomes, intervention-comparators, and indications, comprised our final analysis sample. The median RoR comparing RCT and observational study relative effect estimates was 1.09 (range: 0.15 to 10.82), and the interquartile range was between 0.78 and 1.46. CONCLUSIONS: Overall, our review shows variation in the consistency between observational and RCT treatment effect estimates. Ongoing analysis will evaluate the differences and consistencies, and the impact of observational study designs and methodologies on the consistency between RCT and observational study estimates.
Conference/Value in Health Info
2020-11, ISPOR Europe 2020, Milan, Italy
Value in Health, Volume 23, Issue S2 (December 2020)
Code
PDG1
Topic
Clinical Outcomes
Topic Subcategory
Comparative Effectiveness or Efficacy
Disease
Drugs, No Specific Disease
