Presentation (CTI)

OBJECTIVES: There are often many plausible model structures, assumptions, and data sources for a health economic model. The sensitivity of model results to these methodological choices can be described as ‘modelling approach uncertainty’. Only a subset of alternatives is typically evaluated because programming, executing, and evaluating additional modelling approaches is resource intensive, especially if these involve changes to the model engine. This study assessed the performance of an AI-based pipeline in automatically implementing alternative modelling approaches (including different health state structures, assumptions, and settings), evaluating the plausibility of the resulting base cases, and synthesizing findings to identify key drivers of modelling approach uncertainty.
METHODS: The pipeline was evaluated using two previously published health economic models. For each model, a health economist described alternative model structures, assumptions, and settings to explore, alongside external data sources (e.g., landmark survival estimates) and other criteria to inform plausibility assessment. The pipeline automatically generated the required alternative model implementations by editing the original models and then assessed the resulting base cases against the plausibility criteria, before synthesizing the findings with textual and visual summaries. The accuracy of model implementations and plausibility assessments were evaluated through manual review.
RESULTS: Across both case studies, the pipeline implemented and evaluated over 100 different modelling approaches, synthesizing the findings into a report. Manual review by two health economists found model implementations and plausibility assessments to be highly accurate.
CONCLUSIONS: By overcoming resource constraints associated with traditional evaluation of alternative modelling approaches, generative AI may enable significantly more comprehensive and systematic assessment of modelling approach uncertainty. This could provide decision-makers with a more robust understanding of the impact of modelling choices on estimates of intervention value.