BENCHMARKING AI-SUPPORTED EVIDENCE SYNTHESIS TOOLS FOR HEALTH TECHNOLOGY ASSESSMENT (HTA) EVIDENCE GENERATION: A COMPARATIVE ASSESSMENT ACROSS THE EVIDENCE LIFECYCLE
Author(s)
Rozee Liu, MSc1, Stacy Grieve, PhD1, Sharada Harricharan, PharmD2, Anna Forsythe, MBA, MSc, PharmD1.
1Oncoscope, Miami, FL, USA, 2Frontier HEOR, Toronto, ON, Canada.
1Oncoscope, Miami, FL, USA, 2Frontier HEOR, Toronto, ON, Canada.
OBJECTIVES: HTA evidence generation relies on interconnected activities including systematic literature reviews (SLRs), clinical evidence synthesis, real-world evidence evaluation, and economic modelling. Increasing evidence volumes and frequent updates under HTA and Joint Clinical Assessment (JCA) frameworks have intensified operational burdens. Numerous artificial intelligence (AI)-supported evidence synthesis tools have emerged; however, their ability to support the broader HTA evidence-generation lifecycle remains unclear. This study evaluated AI-supported SLR tools across the HTA evidence-generation (EviGen) pipeline.
METHODS: A scoping review of publicly available AI-supported SLR software was conducted using websites, documentation, white papers, conference abstracts, and reference lists. Tools relevant to oncology HTA were identified and mapped against a predefined EviGen framework. Capabilities were assessed across eight steps spanning evidence-generation: (1) protocol design, (2) search strategy development, (3) screening and review, (4) data extraction, (5) evidence synthesis, (6) risk-of-bias assessment, (7) interpretation, and (8) reporting. Published performance benchmarks were reviewed.
RESULTS: Nine AI-supported HTA tools were identified. Seven supported AI-assisted screening and review, while only a minority supported downstream evidence-synthesis activities. Four tools reported screening performance benchmarks, with sensitivity ranging from 14%-99%, specificity from 46%-99%, and false-negative rates from 21%-39%. No identified tool supported all eight EviGen steps, and none enabled continuous living evidence maintenance. Most solutions automated review tasks rather than end-to-end evidence generation. To address these gaps, a comprehensive Real-Time AI-Assisted Living Systematic Literature Review (REAL-SLR) framework was developed in which evidence identification, screening, extraction, synthesis, and evidence mapping are continuously maintained, while bias assessment, interpretation, and reporting remain project-specific.
CONCLUSIONS: Current AI-supported evidence synthesis tools automate selected SLR tasks but do not support the HTA evidence-generation lifecycle or continuous evidence maintenance. As HTA and JCA increasingly require timely, transparent, and continuously updated evidence, living evidence frameworks integrating AI, governance, and reproducibility may represent the next step in HTA evidence generation.
METHODS: A scoping review of publicly available AI-supported SLR software was conducted using websites, documentation, white papers, conference abstracts, and reference lists. Tools relevant to oncology HTA were identified and mapped against a predefined EviGen framework. Capabilities were assessed across eight steps spanning evidence-generation: (1) protocol design, (2) search strategy development, (3) screening and review, (4) data extraction, (5) evidence synthesis, (6) risk-of-bias assessment, (7) interpretation, and (8) reporting. Published performance benchmarks were reviewed.
RESULTS: Nine AI-supported HTA tools were identified. Seven supported AI-assisted screening and review, while only a minority supported downstream evidence-synthesis activities. Four tools reported screening performance benchmarks, with sensitivity ranging from 14%-99%, specificity from 46%-99%, and false-negative rates from 21%-39%. No identified tool supported all eight EviGen steps, and none enabled continuous living evidence maintenance. Most solutions automated review tasks rather than end-to-end evidence generation. To address these gaps, a comprehensive Real-Time AI-Assisted Living Systematic Literature Review (REAL-SLR) framework was developed in which evidence identification, screening, extraction, synthesis, and evidence mapping are continuously maintained, while bias assessment, interpretation, and reporting remain project-specific.
CONCLUSIONS: Current AI-supported evidence synthesis tools automate selected SLR tasks but do not support the HTA evidence-generation lifecycle or continuous evidence maintenance. As HTA and JCA increasingly require timely, transparent, and continuously updated evidence, living evidence frameworks integrating AI, governance, and reproducibility may represent the next step in HTA evidence generation.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR260
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
Oncology