Artificial Intelligence (AI)-Assisted Early Data Insights and Literature Monitoring: A Case Study of Maintaining an Up-to-Date Reference Library in Metastatic Prostate Cancer (MPC)

Author(s)

Patel V1, Musat M2, Jafar R1, Grieve S1, Rizzo M3, Young V4
1Cytel Inc, Toronto, ON, Canada, 2Cytel Inc, Salem, NH, USA, 3Cytel Inc., Kent, KEN, UK, 4Cytel Inc., London, LON, UK

OBJECTIVES: The application of AI tools for review and extraction of records has the potential to provide early insights during literature monitoring and reduce the burden of maintaining an up-to-date literature review. We aimed to demonstrate the use of AI-assisted screening and extraction for frequent monitoring of studies that meet predefined inclusion criteria, published after the search date of the original literature review.

METHODS: Studies in mPC from eight key oncology congresses reporting clinical, epidemiology, quality of life, and economic outcomes at any line of therapy were monitored from January 2023 to February 2024. Abstracts were sequentially reviewed against the predefined selection criteria, by both an AI model validated on over 65,000 human-annotated records (LiveSTART™) and human reviewers upon publication. Following selection, another AI model, (LiveRef™) was used to extract the category of evidence, study type, and intervention. Specificity and accuracy of the AI reviewer LiveSTART™ and the extraction accuracy of LiveRef™ were assessed by comparing to the human review and extraction.

RESULTS:: Across one year, a total of 15,668 abstracts from 8 congresses were reviewed. The total review time was 135 minutes for LiveSTART™ and 1440 minutes for the human reviewer. Following human review, 427 abstracts were included. The specificity and accuracy of LiveSTART™ versus human review were on average 93% and 92%, respectively. The total extraction time of the 427 records was 60 minutes for LiveRef™ and 1025 minutes for the human reviewer. The extraction accuracy of LiveRef™ was 88% for evidence category, 87% for study type, and 88% for interventions.

CONCLUSIONS: LiveSTART™ and LiveRef™ were used effectively for frequent literature monitoring and performed study screening and extraction with high specificity and accuracy within a significantly shorter timeframe compared to human review. LiveRef™ efficiently extracted key study characteristics, providing early insights from a large volume of abstracts released periodically throughout the year.

Conference/Value in Health Info

2024-11, ISPOR Europe 2024, Barcelona, Spain

Value in Health, Volume 27, Issue 12, S2 (December 2024)

Code

MSR116

Topic

Methodological & Statistical Research, Study Approaches

Topic Subcategory

Artificial Intelligence, Machine Learning, Predictive Analytics, Literature Review & Synthesis

Disease

No Additional Disease & Conditions/Specialized Treatment Areas, Oncology

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×