Presentation (CTI)

OBJECTIVES: Bayesian indirect treatment comparisons (ITCs) are popular in HTA submissions for their flexibility and, importantly, their ability to provide probabilistic statements of uncertainty in comparisons. Best-practices in Bayesian modelling emphasize the importance of prior-predictive checks to ensure credibility of conclusions but this step is generally not conducted in HTA submissions.
METHODS: We use the multinma contrast model and anxiety dataset with default inputs (independent normal(0, 100) on treatment effects) and conduct prior predictive tests assessing plausibility of relative effects on the SMD scale. Comparisons are summarized using medians and 95% credible intervals and predictive intervals. We compare these to alternatives recently suggested in the literature including a version with normal(0,3) priors on SMDs vs the reference, one where all active therapies are modeled hierarchically, and one where all parameters are set to reflect a plausible data-generating mechanism.
RESULTS: Default priors lead to summaries across all domains that lack face validity including: SMDs greater than 10 being more likely than those between -0.5 and 0.5 and prediction intervals that suggest credibility that an SMD of 0.5 could be as large as 11 in a replication trial. Comparisons between active therapies show even greater lack of credibility since uncorrelated priors mean the prior predictive variance of the difference between actives is twice the difference versus control. Tighter priors on treatment effects alone showed minimal benefit, with greater plausibility from class-based and full prior-predictive calibration approaches. Priors used meaningfully change central estimates, ranking summaries, and comparisons between active therapies.
CONCLUSIONS: Probability statements from Bayesian ITCs using the most common approach to setting priors lack credibility and conclusions can meaningfully diverge from those set using best modelling practices. Complete prior information is rarely available but reasonable constraints should be set based on physical and logical constraints.