Is Artificial Intelligence Ready to Tackle Language Barriers for HEOR Research?

OBJECTIVES: Artificial intelligence (AI) has evolved significantly, impacting many areas, including translation services. This study aims to assess the accuracy of AI translation in Health Economics and Outcomes Research (HEOR) publications.

METHODS: Five English-language abstracts from published manuscripts covering various HEOR disciplines, including real-world data, cost-effectiveness analysis, patient-reported outcomes, systematic literature reviews, and randomized controlled trials were selected. Each abstract was translated into Chinese, French, German, Italian, and Japanese using OpenAI’s ChatGPT 3.5. Five HEOR researchers, who were native speakers of these languages, assessed the quality of translations. The review focused on HEOR jargon and whether the translations preserved the original meanings in English. Reviewers also ranked the quality of translated abstracts by type of research. The translated abstracts were also translated back to English (reverse translation) to identify discrepancies with the original abstract.

RESULTS: Overall, reviewers were able to comprehend the main takeaways of the abstracts across all languages except French, where key HEOR terms were often translated inaccurately. Although the methods and results sections were generally translated properly, research jargon including “follow-up”, “efficacy”, “treatment naïve”, “washout”, “dominant”, and “quality-adjusted life years” was not always translated to reflect the original meaning in English, nor maintained due to the lack of equivalent term in the translated language. After reverse translation, the abstracts were very consistent with the original ones with minor missing and inaccurate errors. In Japanese and Chinese, disease and drug names were inconsistently translated to reflect the common terms in those languages. Reviewers' rankings of abstract translation quality varied by type of research, which could be explained by the language structure and the amount of research literature available in the non-English language.

CONCLUSIONS: AI showed promising capabilities in translating HEOR publications from English to other languages. Nevertheless, there are opportunities to further enhance the use of AI translation tools for HEOR professionals.