Reconciling Discrepancies in Results with Different Population-adjusted Indirect Comparison (PAIC) Methods
Author(s)
K. Jack Ishak, PhD1, Conor Chandler, BS, MSc2, Irina Proskorovsky, MSc3, Kyle Fahrbach, PhD2, Venediktos Kapetanakis, PhD4.
1Vice President, Statistical Methodology and Strategy, Thermo Fisher Scientific, St-Laurent, QC, Canada, 2Thermo Fisher Scientific, Waltham, MA, USA, 3Thermo Fisher Scientific, Madrid, Spain, 4Thermo Fisher Scientific, London, United Kingdom.
1Vice President, Statistical Methodology and Strategy, Thermo Fisher Scientific, St-Laurent, QC, Canada, 2Thermo Fisher Scientific, Waltham, MA, USA, 3Thermo Fisher Scientific, Madrid, Spain, 4Thermo Fisher Scientific, London, United Kingdom.
OBJECTIVES: Recent simulation studies have produced conflicting findings about the accuracy of matching-adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and multi-level network meta-regression (ML-NMR) of time-to-event or binary outcomes. Differences in performance of the methods have been debated and may be attributed to the different estimands (marginal vs. conditional) targeted with each. We use a case study to illustrate this phenomenon and highlight where the differences arise.
METHODS: We applied the three methods to compare progression-free-survival with lenalidomide (index) and thalidomide in newly diagnosed multiple-myeloma. Analyses were anchored on placebo and adjusted for imbalances in four potential effect-modifiers (EMs) to reflect comparisons in the thalidomide population. We derive marginal relative effects by contrasting the adjusted effect of lenalidomide vs. placebo (LvP) in the thalidomide population and the relative effect of the two active treatments (LvT) with MAIC and marginalized STC, and conditional relative effects from STC and ML-NMR evaluated at means of EMs. Note that STC at means is a mixed comparison since the adjusted LvP is a conditional effect contrasted with a marginal effect for the comparator.
RESULTS: The adjusted log hazard ratio (LHR) of LvP in the thalidomide population was -0.51 with MAIC, consistent with the estimated LHR from marginalized STC. These differed from conditional results with STC and ML-NMR at means, which yielded LHRs of -0.60 and -0.61, respectively. Similarly, LvT comparisons from MAIC aligned with marginalized STC (LHR=-0.41), while the conditional effects were slightly weaker (LHR=-0.5). Standard errors for LvP and LvT were consistent across analyses.
CONCLUSIONS: Differences in results in this case study were explainable by the type of effect being estimated. Although the overall conclusion was unchanged with marginal and conditional effects, the magnitude of effects did differ. It is, therefore, important to clearly define the targeted comparison and apply PAIC methods accordingly for valid inference.
METHODS: We applied the three methods to compare progression-free-survival with lenalidomide (index) and thalidomide in newly diagnosed multiple-myeloma. Analyses were anchored on placebo and adjusted for imbalances in four potential effect-modifiers (EMs) to reflect comparisons in the thalidomide population. We derive marginal relative effects by contrasting the adjusted effect of lenalidomide vs. placebo (LvP) in the thalidomide population and the relative effect of the two active treatments (LvT) with MAIC and marginalized STC, and conditional relative effects from STC and ML-NMR evaluated at means of EMs. Note that STC at means is a mixed comparison since the adjusted LvP is a conditional effect contrasted with a marginal effect for the comparator.
RESULTS: The adjusted log hazard ratio (LHR) of LvP in the thalidomide population was -0.51 with MAIC, consistent with the estimated LHR from marginalized STC. These differed from conditional results with STC and ML-NMR at means, which yielded LHRs of -0.60 and -0.61, respectively. Similarly, LvT comparisons from MAIC aligned with marginalized STC (LHR=-0.41), while the conditional effects were slightly weaker (LHR=-0.5). Standard errors for LvP and LvT were consistent across analyses.
CONCLUSIONS: Differences in results in this case study were explainable by the type of effect being estimated. Although the overall conclusion was unchanged with marginal and conditional effects, the magnitude of effects did differ. It is, therefore, important to clearly define the targeted comparison and apply PAIC methods accordingly for valid inference.
Conference/Value in Health Info
2025-05, ISPOR 2025, Montréal, Quebec, CA
Value in Health, Volume 28, Issue S1
Code
MSR141
Topic
Methodological & Statistical Research
Disease
No Additional Disease & Conditions/Specialized Treatment Areas