OBJECTIVES: The objective of this study was to conduct a feasibility assessment for using Chat-GPT to complete Quality Assessment (QA) questionnaires.

METHODS: A custom Chat-GPT-4 application was developed in Python 3 to take inputs from Rinvoq study/trial reports and the ROB 2.0 questionnaire to produce a completed assessment. The questionnaires were also completed by experienced systematic review experts. The two sets of questionnaires were compared to assess accuracy and estimate time savings. All data were reviewed to identify opportunities and limitations in using generative artificial intelligence (AI) for completing QA tools.

RESULTS: The custom Chat-GPT application's output for QA tools was 90-95% accurate. The quantitative questions were completed with 95-100% accuracy. The time savings for completing the ROB 2.0 questionnaire averaged 15.5 hours per study. However, Chat-GPT was unable to successfully complete two subjective questions related to methods for analysis and reporting of outcomes. These subjective questions required highly experienced HEOR experts to review study reports and search for additional supporting documents to correctly complete those sections. Overall, Chat-GPT shows promise in saving significant time and resources.

CONCLUSIONS: Custom Chat-GPT applications have the potential to save significant time, effort, and monetary resources. However, outputs from these AI tools need to be corrected and/or validated by experienced HEOR experts.