Artificial Intelligence (AI)-Assisted Early Data Insights and Literature Monitoring: A Case Study of Maintaining an Up-to-Date Reference Library in Metastatic Prostate Cancer (MPC)

OBJECTIVES: The application of AI tools for review and extraction of records has the potential to provide early insights during literature monitoring and reduce the burden of maintaining an up-to-date literature review. We aimed to demonstrate the use of AI-assisted screening and extraction for frequent monitoring of studies that meet predefined inclusion criteria, published after the search date of the original literature review.

METHODS: Studies in mPC from eight key oncology congresses reporting clinical, epidemiology, quality of life, and economic outcomes at any line of therapy were monitored from January 2023 to February 2024. Abstracts were sequentially reviewed against the predefined selection criteria, by both an AI model validated on over 65,000 human-annotated records (LiveSTART™) and human reviewers upon publication. Following selection, another AI model, (LiveRef™) was used to extract the category of evidence, study type, and intervention. Specificity and accuracy of the AI reviewer LiveSTART™ and the extraction accuracy of LiveRef™ were assessed by comparing to the human review and extraction.

RESULTS:: Across one year, a total of 15,668 abstracts from 8 congresses were reviewed. The total review time was 135 minutes for LiveSTART™ and 1440 minutes for the human reviewer. Following human review, 427 abstracts were included. The specificity and accuracy of LiveSTART™ versus human review were on average 93% and 92%, respectively. The total extraction time of the 427 records was 60 minutes for LiveRef™ and 1025 minutes for the human reviewer. The extraction accuracy of LiveRef™ was 88% for evidence category, 87% for study type, and 88% for interventions.

CONCLUSIONS: LiveSTART™ and LiveRef™ were used effectively for frequent literature monitoring and performed study screening and extraction with high specificity and accuracy within a significantly shorter timeframe compared to human review. LiveRef™ efficiently extracted key study characteristics, providing early insights from a large volume of abstracts released periodically throughout the year.