Man Versus Machine: Can AI-Assisted Technology be Used to Support the Development of Economic Models?

Author(s)

Medland S1, Davies H2, Butler K3, Moss J2
1York Health Economics Consortium Ltd, Cardiff, CRF, UK, 2York Health Economics Consortium Ltd, York, York, UK, 3York Health Economics Consortium Ltd, York, YOR, UK

OBJECTIVES: This research aims to explore whether an artificial intelligence assisted technology (AI) can be utilized for writing and reviewing standardized functions in Visual Basic for Applications (VBA) for health economic models (HEM) built in Microsoft Excel.

METHODS: A fully incremental analysis (FIA) is required for health technology evaluations submitted to NICE to determine whether technologies are dominated or extendedly dominated. An AI was asked to write a VBA function to automate an FIA for a HEM. Feedback was provided to the AI and new code was generated three times. Each iteration was timed, reviewed and results cross-checked with an example model. Additionally, a modeler created a bespoke VBA function to automate an FIA. The AI was asked to review the code and make it more efficient. The changes made to the code by the AI were reviewed by a modeler and results were validated.

RESULTS: The AI required a specific research question to ensure the function returned the appropriate outcomes. For each iteration, the code and accompanying guidance was generated in under two minutes. Minor issues with the code included use of hardcoded values; missing arguments; incorrectly defined variables; use of key words not used in VBA (e.g. continue for), and outputting results in an incorrect format. Two iterations correctly determined dominance, but extended dominance calculations were incorrect in all iterations. The third iteration contained a calculation error causing incorrect results. When asked to review a modeler’s function, the AI suggested several valid adjustments; however, the new code was poorly styled, and some key calculations were incorrect.

CONCLUSIONS: Code generated by AI may improve in future if more opensource model code is published. Currently, AI cannot replace the expertise and experience of modelers but has the potential to be a useful tool to review code and give advice.

Conference/Value in Health Info

2023-11, ISPOR Europe 2023, Copenhagen, Denmark

Value in Health, Volume 26, Issue 11, S2 (December 2023)

Acceptance Code

P40

Topic

Methodological & Statistical Research

Topic Subcategory

Artificial Intelligence, Machine Learning, Predictive Analytics

Disease

no-additional-disease-conditions-specialized-treatment-areas

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×