Presentation (CTI)

OBJECTIVES: Uncritical use “off the shelf” network meta-analysis models without data exploration and consideration of modifications may overestimate heterogeneity and lead to inappropriate conclusions or use of more complex population adjustment methods. This research explores the impact of poor modelling practices in a case study in plaque psoriasis.
METHODS: We use an example of a large network in moderate to severe plaque psoriasis to examine how assumptions of proportional odds across thresholds in the baseline arm leads to borrowing of information across studies, and compare this to an a model where each study’s baseline treatment has individual intercepts estimated. We further examine the influence of logit compared to probit link. Models are compared in terms of absolute (total residual deviance) and relative fit (DIC), in addition to the magnitude of the baseline risk adjustment beta and between trial standard deviation.
RESULTS: Choice of logit link lead to smaller total residual deviance (831 vs 924 on 370 data points) but the between trial standard deviation was larger for logit (0.32) than probit (0.19) and DIC was identical (2516). Betas for both model were similar and excluded null (-1.02 vs -1.01). Models with trial specific intercepts showed improved absolute and relative fit favouring logit link (totresdev = 443 vs 480; DIC = 2192 vs 2231). Between trial standard deviation (0.12 and 0.08) and baseline risk betas (-0.21 vs -0.11) shrunk considerably. Reduction in heterogeneity is so large that a fixed effect model may be preferred based on DIC (DIC diff = 3).
CONCLUSIONS: Uncritical application of standard network meta-analysis models may lead to inflated standard errors, hide treatment differentiation. Poor modelling may cause unnecessary use of population adjusted models that decrease transparency and replicability, and increase the complexity of decision-making.