Quantifying the Impact of Exposure Prevalence and Selection Bias to Optimize Patient Selection Criteria in Post-Authorization Safety Studies

OBJECTIVES: Regulatory agencies are increasingly requiring retrospective observational studies to assess the risk of major congenital malformations (MCM) following prenatal exposures. Obtaining sufficient sample size of patients with the indication and prenatal exposure to rule out increased risk of MCM can be challenging; modest changes to the patient selection criteria will affect the study’s power to detect differences in exposed and unexposed. The objectives of this analysis were to identify sample sizes where changes to patient selection criteria would impact the ability to detect an increased risk of MCM and assess how sensitive the estimated risk of MCM is to potential selection bias.

METHODS: A power analysis was conducted to determine the minimum sample size necessary to detect a two-fold increase in relative risk (RR) of MCM with a two-sided α=0.05 and 80% power. This analysis assumed a 4% rate of MCM in unexposed infants and varied exposure prevalence from 5-40%. Simulations were conducted to evaluate the sensitivity of the RR estimate to selection bias for different exposure prevalence and sample sizes.

RESULTS: Sample size needed to detect a two-fold increase in RR was inversely related to exposure prevalence and varied from 5,114 with a 5% prevalence to 1,132 with a 40% prevalence. With moderate exposure prevalence (20-40%), reductions in sample size of less than 200 resulted in failure to detect two-fold differences in RR of MCM. In scenarios where moderate selection bias was assumed, even smaller reductions in sample size resulted in failure to correctly detect a two-fold difference in RR.

CONCLUSIONS: Though alterations to patient selection criteria may have only modest effects on the total patients included in the study, these changes can impact statistical power to detect differences in the RR. The ability to correctly detect differences is further decreased when the alterations to patient selection criteria introduce selection bias.