TREATMENT EFFECT ESTIMATION- A COMPARISON BETWEEN RANDOMIZED CONTROLLED TRIALS AND OBSERVATIONAL STUDY WITH THREE DIFFERENT CAUSAL EFFECT METHODS-PROPENSITY SCORE, MARGINAL STRUCTURE MODEL AND DOUBLY ROBUST METHODS
Author(s)
Peter Sun, MD, PhD, Research Scientist1, Don Buesching, PhD, Research Advisor2, Robert Obenchain, PhD, Senior Research Advisor21Lilly Research Laboratories, Indianapolis, IN, USA; 2 Eli Lilly and Company, Indianapolis, IN, USA
OBJECTIVES: To examine whether results from two small randomized controlled trials (RCTs) converge with the results from a large observational study with causal effect methods (OSCEMs) in comparing the treatment effects of 2 insulin regimens, and to compare different causal effect methods including propensity score (PSM), marginal structure model (MSM) and doubly robust methods (DRM). METHODS: We first theoretically examined the similarity and difference as well as pros and cons of RCT and OSCEM approaches in TE estimation. Then, 2 RCTs (n=105 and 97; crossover at 16th weeks) and 3 large OSCEMs (n=4,519, in 4 post-baseline quarters, propensity score, marginal structural model and doubly robust methods) were conducted to examine whether the differences of treatment effect between 2 insulin regimens estimated through RCTs converge with the differences of the same treatment effect estimated through 3 OSCEMs. The treatment effect was defined as the reduction of hemoglobin A1c (HbA1c) among patients with type 2 diabetes. The 2 insulin regimens were once-daily basal analog insulin (QDBAI)---glargine vs. twice-daily premixed analog insulin (BIDMAI)---lispro mix 75/25. RESULTS: Theoretically, RCTs have higher internal validity and lower external validity compared to OSCEMs. The differences of HbA1c reductions between BIDMAI and QDBAI in 4 quarters estimated through 2 RCTs were similar to the differences estimated through 3 OSCEMs (RCTs: -0.39---p=0.02, -0.38---p=0.03 vs. OSCEMs: -0.48---p<0.01, -0.72---p=0.12, -0.31---p=0.11). CONCLUSION: Compared to OSCEMs, RCTs may have higher internal validity, but lower external validity. With correct exercises, the results of OSCEMs and RCTs can converge. OSCEMs are larger sample methods that may have better external validity, but require a large sample size. Compared to PSM, MSM and DRM are less efficient, may require an even larger sample size.
Conference/Value in Health Info
2007-05, ISPOR 2007, Arlington, VA, USA
Value in Health, Vol. 10, No.3 (May/June 2007)
Code
DB3
Topic
Clinical Outcomes
Topic Subcategory
Clinical Outcomes Assessment
Disease
Diabetes/Endocrine/Metabolic Disorders