A Step-By-Step Guide for Causal Study Design When Estimating Treatment Effects Using Real-World Data


Hoffman S1, Gangan N1, Chen X1, Smith JL1, Tave A1, Yang Y1, Dosreis S2, Grabner M1
1HealthCore, Inc., Wilmington, DE, USA, 2University of Maryland School of Pharmacy, Baltimore, MD, USA

OBJECTIVES: Causal inference (CI) in observational research is growing more important, driven by the need for generalizable and rapidly delivered real-world evidence (RWE) to inform regulatory, payer, and patient/provider decision-making. Existing methodological literature on this topic is rich but can be complex and daunting to navigate. We describe a framework to approach these methods with confidence.

METHODS: A team with diverse training and research backgrounds developed a visual “step-by-step guide to causal study design,” and corresponding glossary, after a comprehensive review of CI literature. During this process, we identified and addressed key conceptual issues that researchers should be aware of to develop confidence in implementing CI methods.

RESULTS: We describe eight steps in causal study design. Step 1: Define an explicitly causal research question. Step 2: Decide on a “first treatment carried forward” (analogous to “intention to treat”) or an “as-treated” (analogous to “per protocol”) outlook. Step 3: Define the population for the counterfactual contrast, e.g., average treatment effect in the entire study population or in the treated. Step 4: Select a measure of effect, e.g., rate or risk, difference or ratio. Together, steps 2-4 define the study estimand. Step 5: Develop a directed acyclic graph for the research question and estimand to clarify the causal pathway. Step 6: Identify potential biases, e.g., immortal time. Step 7: Select appropriate design elements and statistical methods to evaluate and address potential biases. Step 8: Conduct sensitivity analyses to assess robustness of the methods used and quantify remaining biases, e.g., unmeasured confounding.

CONCLUSIONS: We identified and described key conceptual issues of importance to researchers who design CI studies, and created a visually appealing, user-friendly resource to help researchers clearly define and navigate these issues. This guidance serves to enhance the quality and thus the impact of RWE from observational research.

Conference/Value in Health Info

2022-05, ISPOR 2022, Washington, DC, USA

Value in Health, Volume 25, Issue 6, S1 (June 2022)




Clinical Outcomes, Methodological & Statistical Research, Study Approaches

Topic Subcategory

Comparative Effectiveness or Efficacy, Confounding, Selection Bias Correction, Causal Inference, Electronic Medical & Health Records


No Additional Disease & Conditions/Specialized Treatment Areas

Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now