OBJECTIVES: Real-world data (RWD) can facilitate drug safety studies including rapid signal assessments, but may be prone to confounding by indication due to non-randomization. We used immune checkpoint inhibitor (ICI)-related cholangitis as a case study and compared results from a quasi-experimental design with those from conventional observational study designs.

METHODS: Data were derived from PharMetrics®, a US insurance database, from 2011 to 2022. A difference-in-difference (DID) design was applied, incorporating 2 cancer cohorts: an ICI-eligible cohort not treated with ICIs, followed over their 2 most recent years of cancer diagnosis, and an ICI-treated cohort followed from 12 months before to 12 months after ICI initiation. Cholangitis was identified using International Classification of Diseases codes. The hazard ratio (HR) of cholangitis associated with ICI therapy was derived from a Cox proportional hazards model using an interaction term between cohort and year of follow-up. Results were compared with those from unadjusted and adjusted analyses of the cohort study design and the pre-post cohort study design.

RESULTS: Of the 472,408 patients included, 54% were male; mean (SD) age was 63 (12) years. The non-ICI cohort had an incidence rate (IR) of 1.67 (95% CI: 1.55-1.80) per 1,000 person-years in the first year of follow-up and 3.86 (3.68-4.05) in the second year. In the ICI-treated cohort, IR was 2.88 (2.29-3.61) pre-treatment and 5.15 (4.16-6.38) post-treatment. While ICI-therapy was not associated with cholangitis (HR: 0.91 [0.66-1.26]) in the DID study, a statistically significant association was observed in the cohort study and the pre-post cohort study (HR ranging from 1.39 to 1.93, p<0.006), even after adjustment by inverse probability weighting.

CONCLUSIONS: Confounding by indication is a key threat to the internal validity of observational real-world drug safety studies. Our findings show that a DID design offers a practical approach with the potential to mitigate bias in the absence of exchangeability.