Automating Economic Modeling: Potential of Generative AI for Updating Modeling Reports


Rawlinson W1, Teitsson S2, Reason T3, Malcolm B2, Gimblett A1, Klijn S4
1Estima Scientific Ltd, London, UK, 2Bristol Myers Squibb, Uxbridge, UK, 3Estima Scientific Ltd, South Ruislip, LON, UK, 4Bristol-Myers Squibb, Utrecht, ZH, Netherlands

Presentation Documents

OBJECTIVES: Using large language models (LLMs) such as Generative Pre-trained Transformer 4 (GPT-4) to edit Microsoft Word files could revolutionize the reporting of health economic models. This study aimed to assess GPT-4’s capabilities in automatically updating a Word technical report for a cost-utility model that was used in health technology assessments (HTAs) for muscle-invasive urothelial carcinoma (MIUC).

METHODS: The MIUC model was first manually edited. Then, utilizing GPT-4 and deterministic programming, the Word technical report was automatically updated to reflect the Excel model. GPT-4 updated text in the results and conclusion sections, and all automated edits were captured in tracked changes. Two experienced health economists then blindly assessed the AI-adapted report alongside a manually adapted report that was developed by a third health economist. Accuracy was evaluated based on correct/incorrect changes, correct/incorrect retainment of original text, and instances of missing information.

RESULTS: Both reviewers identified more than 30 instances for scoring. Accuracy was 94.3% for the AI-adapted report and 98.5% for the manually adapted report. The reviewers agreed there were 2 incorrect changes in the AI-adapted report: a rounding error and an incorrect description of a scenario analysis. Qualitatively, the reviewers generally approved of the tone of edits made by GPT-4. However, there were a small number of factually correct edits where the reviewers preferred language chosen by the human health economist.

CONCLUSIONS: This study is a promising early indication that LLMs can be leveraged as a part of a reviewer-friendly pipeline for automatically updating model technical reports in Microsoft Word. The accuracy achieved in our study suggests suitability as a first editor prior to human review. Utilizing AI to adapt technical reports for HTA submissions could accelerate dissemination of health technologies around the world.

Conference/Value in Health Info

2024-05, ISPOR 2024, Atlanta, GA, USA

Value in Health, Volume 27, Issue 6, S1 (June 2024)




Economic Evaluation, Methodological & Statistical Research

Topic Subcategory

Artificial Intelligence, Machine Learning, Predictive Analytics, Cost-comparison, Effectiveness, Utility, Benefit Analysis


No Additional Disease & Conditions/Specialized Treatment Areas, Oncology

Explore Related HEOR by Topic

Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on Update my browser now