Parametric Model Selection: Beyond AIC/BIC in Health Economic Context
Author(s)
Beatrice Suero1, Roya Gavanji, MSc2, Aidan Dineen, PhD2.
1Associate Director, HEOR, EVERSANA, Oakville, ON, Canada, 2EVERSANA, Burlington, ON, Canada.
1Associate Director, HEOR, EVERSANA, Oakville, ON, Canada, 2EVERSANA, Burlington, ON, Canada.
OBJECTIVES: The Health Technology Assessment (HTA) submission should include a detailed assessment of the goodness of fit for the parametric models and justification for the choice of model used to extrapolate the time to event outcomes. In this abstract, we review methods to assess model suitability.
METHODS: Submissions to HTA agencies routinely require time to event (eg, survival) analyses, including parametric survival models, to inform health economic models. Although AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) are common model selection metrics, they do not always guarantee the best model, particularly at the tail where data are sparse. Additional factors such as visual inspection of survival curves, residual plots, proportional hazards and accelerated failure time assumption tests, and assessment of clinical plausibility were evaluated in different scenarios to select the best parametric model.
RESULTS: The findings indicate that the choice of the best model should ideally involve assessing the predicted survival both visually and by examining model fit statistics and plots. Estimated survival can be sensitive to model choice. Therefore, the selection of an appropriate model is critical to guide decision-making. Clinical experts’ opinion should also support model choice, particularly when the data are immature and extrapolated tails are varied across models. Clinical experts can provide input on the shape of the hazard functions projected for the long-term and support the choice of model.
CONCLUSIONS: When selecting the best parametric model, there are various criteria that should be considered in addition to AIC and BIC. The evidence herein highlights that by leveraging approaches including visual inspection, model performance, and model assumption tests, the most appropriate, reliable and interpretable model is chosen for the data. Supplemental feedback from clinical experts can improve the clinical plausibility of survival extrapolations based on the selected model.
METHODS: Submissions to HTA agencies routinely require time to event (eg, survival) analyses, including parametric survival models, to inform health economic models. Although AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) are common model selection metrics, they do not always guarantee the best model, particularly at the tail where data are sparse. Additional factors such as visual inspection of survival curves, residual plots, proportional hazards and accelerated failure time assumption tests, and assessment of clinical plausibility were evaluated in different scenarios to select the best parametric model.
RESULTS: The findings indicate that the choice of the best model should ideally involve assessing the predicted survival both visually and by examining model fit statistics and plots. Estimated survival can be sensitive to model choice. Therefore, the selection of an appropriate model is critical to guide decision-making. Clinical experts’ opinion should also support model choice, particularly when the data are immature and extrapolated tails are varied across models. Clinical experts can provide input on the shape of the hazard functions projected for the long-term and support the choice of model.
CONCLUSIONS: When selecting the best parametric model, there are various criteria that should be considered in addition to AIC and BIC. The evidence herein highlights that by leveraging approaches including visual inspection, model performance, and model assumption tests, the most appropriate, reliable and interpretable model is chosen for the data. Supplemental feedback from clinical experts can improve the clinical plausibility of survival extrapolations based on the selected model.
Conference/Value in Health Info
2025-05, ISPOR 2025, Montréal, Quebec, CA
Value in Health, Volume 28, Issue S1
Code
MSR3
Topic
Methodological & Statistical Research
Disease
No Additional Disease & Conditions/Specialized Treatment Areas