Extending External Validity of Rcts and Meta-Analysis through Transportability Analysis and Integration of Mechanistic Research Evidence

OBJECTIVES: Randomized clinical trials (RCTs)and meta-analyses of RCTs are placed at the top of the evidence hierarchy because of their high internal validity. However, RCTs often suffer from limited external validity. Evidence synthesis methods that integrate mechanistic evidence are needed to better understand the extent to which RCT findings apply to diverse clinical settings and populations. Transportability analysis is a causal inference method that assesses the transferability of findings from a study population to a distinct target population. Here, we assess the feasibility of using transportability analysis to integrate mechanistic evidence into evidence synthesis.

METHODS: We reviewed evidence-based medicine (EBM) frameworks for evidence synthesis, and methods for extending external validity of RCTs and meta-analyses. To assess feasibility, we studied the national roll-out of efaverinz in Zimbabwe, which was informed by data from trials conducted outside Zimbabwe, leading to significant rates of adverse events and drug discontinuation. We used directed acyclic graph (DAG) reasoning to identify causal relationships and potential effect modifiers between efavirenz and adverse events.

RESULTS: Current EBM frameworks do not provide methods for integrating mechanistic evidence into evidence synthesis. Through DAG reasoning, we identified genetic mutations as an effect modifier of the relationship between efavirenz use and adverse events. Review of data on population-level differences in the prevalence of key genetic mutations at the time of the efavirenz roll-out found sufficient data to conduct transportability analysis. Transportability analysis could have predicted the high rate of adverse events due to efavirenz in Zimbabwe.

CONCLUSIONS: Although RCTs are viewed as the gold standard, they often have limited external validity. The integration of mechanistic research evidence with RCT results is crucial for informed decision-making. Transportability analysis offers a practical approach to bridging this gap, and thus enhancing the applicability of clinical trial findings to different populations and healthcare systems.