This study aimed to provide an overview of analytical methods in scientific literature for comparing uncontrolled medicine trials with external controls from individual patient data real-world data (IPD-RWD) and to compare these methods with recommendations made in guidelines from European regulatory and health technology assessment (HTA) organizations and with their evaluations described in assessment reports.
A systematic literature review (until March 1, 2023) in PubMed and Connected Papers was performed to identify analytical methods for comparing uncontrolled trials with external controls from IPD-RWD. These methods were compared descriptively with methods recommended in method guidelines and encountered in assessment reports of the European Medicines Agency (2015-2020) and 4 European HTA organizations (2015-2023).
Thirty-four identified scientific articles described analytical methods for comparing uncontrolled trial data with IPD-RWD–based external controls. The various methods covered controlling for confounding and/or dependent censoring, correction for missing data, and analytical comparative modeling methods. Seven guidelines also focused on research design, RWD quality, and transparency aspects, and 4 of those recommended analytical methods for comparisons with IPD-RWD. The methods discussed in regulatory (n = 15) and HTA (n = 35) assessment reports were often based on aggregate data and lacked transparency owing to the few details provided.
Literature and guidelines suggest a methodological approach to comparing uncontrolled trials with external controls from IPD-RWD similar to target trial emulation, using state-of-the-art methods. External controls supporting regulatory and HTA decision making were rarely in line with this approach. Twelve recommendations are proposed to improve the quality and acceptability of these methods.
This research is important because it examines how clinical trials with no comparative group can be compared to external groups using individual patient data from real-world settings. These comparisons are increasingly necessary for regulatory and reimbursement decisions regarding new treatments. The study reveals a significant gap between advanced analytical methods discussed in academic literature and those used in practice. While guidelines and scientific literature endorse sophisticated methods, regulatory assessments often rely on simpler, less transparent approaches.
The review highlights several key challenges when using external comparative groups, such as data quality issues, biases, and the complexity of methodologies. It underscores the importance of engaging stakeholders early in the process, providing better guidance, and establishing a more consistent approach to enhance the rigor and acceptance of trials that make comparisons with external real-world data. Current guidelines do not specify when these are acceptable. However, the literature emphasizes the need for high-quality real-world data and suggests that different methods—like matching, weighting, and other advanced statistical techniques—should be employed to account for potential confounding factors.
There are notable variations in the guidelines provided by different organizations. Some focus more on research design than on the specific analytical methods to be used. While individual patient data is generally preferred for building external comparator groups, many reports still rely on aggregated data, which may not meet the rigorous standards set out in the guidelines.
Transparency is another critical issue, as regulatory reports frequently lack clear descriptions of the analytical methods applied, making it hard to evaluate the reliability of their findings. The study stresses the need for continuous dialogue among all parties involved to tackle uncertainties in the use of real-world data.
The review culminates in 12 recommendations aimed at improving the quality and acceptability of analytical methods in regulatory and reimbursement submissions: the necessity for a priori defined protocols, clarity in research questions, and the importance of testing the results in different contexts. Overall, the study calls for a more standardized approach to the use of real-world data in practice to ensure comparator groups are built according to the highest methodological standards.