Presentation (CTI)

OBJECTIVES: ModeLynx is a new health economic modelling application that uses generative AI (GenAI) to build web-based (Python) and Excel cost-effectiveness models (CEMs) and budget impact models (BIMs). However, no published evaluations of its accuracy or fidelity are available. Here we provide an assessment using published CEMs and BIMs to evaluate how well these can be reproduced with different levels of user-input.
METHODS: Three published pharmacoeconomic models were selected for use in this evaluation from a high-quality health-economics journal: one BIM and two CEMs (a Markov model [MM] and a partitioned survival model [PSM]). Publications with supplementary materials were uploaded to ModeLynx and three methods were assessed: minimum user input, guided input using source materials, and extensive additional research. Model accuracy and fidelity were assessed based on replication of results and model structure.
RESULTS: Of the three models, the BIM was most easily reproduced. The relatively straightforward structure meant that minimal efforts were needed to replicate both the web-based and Excel models. Transcription of inputs and generation of outputs were accurate, with the web-based model taking ~1 hour and the Excel ~3 hours. The next most easily reproduced was the MM. Additional guidance was required for PSA, DSA, and scenario analysis, with ~4 hours for the web-based model required and ~6 hours for the Excel. The most difficult to implement was the PSM. The main complexity revolved around efficient implementation of the survival analysis. Here, successful implementation took ~8 and ~16 hours for the web-based and Excel models, respectively.
CONCLUSIONS: ModeLynx provides the opportunity to generate both web-based (Python) and Excel models quickly and efficiently. However, they still need to be carefully reviewed and validated to account for limitations with GenAI. Optimal approaches must include expert modelers and subject-matter experts.