A Systematic Review of NON-SMALL Cell LUNG Cancer Clinical Trial Literature: Robots Versus Humans
Author(s)
Queiros L1, Witzmann A2, Bednarski M3, Sumner M4, Baehrens D4, Abogunrin S1
1F. Hoffmann La Roche, Basel, Switzerland, 2F. Hoffmann La Roche, Kaiseraugst, Switzerland, 3Roche Polska Sp. z o.o., Warsaw, Poland, 4Averbis GmbH, Freiburg, Germany
Background: Health technology agencies require systematic literature reviews (SLRs) of clinical, cost-effectiveness, and humanistic data to make informed decisions. SLRs involve considerable resources–budget and time and so, there is a need to identify processes that will shorten their duration and make them less labour-intensive. To address this gap, we conducted automated reviews of abstracts using two advanced analytic methods (AAMs) to determine their accuracy and correctness compared to humans. Methods: Abstract records from a human-conducted SLR were reviewed by two AAMs. AAM-1 was an adaptation of a pre-trained model designed for mining and classifying biomedical literature and AAM-2 combined support vector machine algorithms with other advanced analytic methodology. A random subset of labelled abstracts was used to train each AAM, following which the remaining unlabeled test abstracts were reviewed by the AAMs. Each AAM automatically and separately assigned an include/exclude status, with reasons, to each record and results were compared to those of the human-conducted SLR. Results: Using predefined SLR protocol criteria, humans included 440 (8%) of 5820 records and excluded the remaining. Similarly, AAM-1 rightly included 6% of records and excluded 79%. Of those excluded, the following number of exclusion reasons were correctly assigned: Population=1671/3047, Intervention=185/303, Outcomes=137/181, and Study design=873/1849. AAM-2 also rightly included 6% of records and excluded 82%, and 1627/3047, 75/303, 22/181 and 1156/1849 were correctly excluded as Population, Intervention, Outcome and Study design, respectively. The time to review completion was 144 hours less compared to humans (191 hours) for either AAM (47 hours). Conclusion: The use of AAMs can alleviate the burden of abstract record reviewing and consequently, potentially free up at least 72% of analysts’ time during the conduct of SLRs. Methods similar to AAMs should be assessed in future research for how consistent their performances are in SLRs of economic, epidemiological and humanistic evidence.
Conference/Value in Health Info
2020-11, ISPOR Europe 2020, Milan, Italy
Value in Health, Volume 23, Issue S2 (December 2020)
Code
PNS218
Topic
Health Technology Assessment, Methodological & Statistical Research, Organizational Practices
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Best Research Practices, Decision & Deliberative Processes
Disease
No Specific Disease