Presentation (CTI)

OBJECTIVES: Indirect treatment comparisons (ITCs) commonly exclude phase 2 studies but can be challenged in peer-review when those studies meet all other PICOS criteria. This research assesses reasons phase 2 studies may systematically differ from phase 3 results and provides considerations for when they should and should not be included in efficacy ITCs.
METHODS: We provide a methodological theory assessment of the main ways in which phase 2 studies differ from phase 3 studies when all other aspects of PICOS are identical. Theoretical concepts are illustrated numerically to demonstrate mechanisms of bias across four main areas of concern: (1) Sample size limitations; (2) Geographic/site heterogeneity; (3) Divergence in outcome hierarchy (primary vs. secondary endpoints); and (4) Selection bias due to "gating," where subsequent initiation of a phase 3 study is contingent on positive phase 2 results.
RESULTS: Numerical illustrations indicate that trial size alone is not sufficient justification for exclusion provided event rates allow for nominal confidence intervals; however, validity is compromised when phase 2 studies exhibit zero-event cells. Geographic limitations threaten validity only if baseline rates or background therapies function as effect modifiers across regions. Differences in outcome hierarchy risk selective reporting bias, where phase 2 outcomes are reported conditionally on significance. Finally, trial gating introduces a complex survivorship bias where in extreme cases (eg, comparing phase 2 data of a candidate therapy against phase 3 data of approved comparators) it creates a structural optimistic bias favoring the candidate therapy.
CONCLUSIONS: Exclusion of phase 2 studies from ITCs when studies otherwise meet PICOS criteria is justified under certain conditions. Routine inclusion of mixed phase studies should include a specific feasibility component related to the four key concepts described in this study. Particular caution should be exercised in “early ITCs” for decision-making that rely on phase 2 data for the focal therapy.