Pilot Study to Evaluate Efficiency of DISTILLERSR®'S Artificial Intelligence (AI) Tool over Manual Screening Process in Literature Review
Author(s)
Kamra S1, Hyderboini R2, Sirumalla Y3, Venkateswara Rao J2, Chidirala S2, Dabral S2, Gogna S2, Mandlik R2, Goyal R4
1IQVIA, Gurugram, HR, India, 2IQVIA, Mumbai, DL, India, 3IQVIA, Gurgaon, India, 4IQVIA, Gurugram, India
Presentation Documents
Objective: This research aims to compare the efficiency (speed and accuracy) of title/abstract (Ti/Ab) screening between manual approach and semi-automated approaches of DistillerSR®. Methods: Embase® electronic database using OvidSP® was searched on 26 August 2021 from 2018 to assess efficacy and safety of anti-TNF agents and their biosimilars in adult patients with ulcerative colitis. Identified references were screened using three approaches: manual, semi-automated screening with DistillerSR® using classifier and AI. Manual screening was done by a single reviewer in MS Excel® DistillerSR® screening using classifier approach: An include/exclude classifier (similar distribution) using sample dataset (~400 references) was created to train classifier. Validity of classifier was confirmed using F1 score. DistillerSR® screening using AI approach: Distiller AI was trained with ~300 references (using quality checked Ti/Ab screening). AI feature generated “AIReview score” for individual references and screened the remaining references. Results: Overall 1257 references were screened using aforementioned methods. In terms of time-saving, AI and classifier approach were more effective than manual screening. AI approach and classifier approach were almost as accurate as manual screening. Total error rate of AI approach and classifier approach were not significantly more than the manual approach (critical error rate was very low using classifier). Conclusion: DistillerSR® screening using classifier approach can be considered a reliable resource in terms of accuracy for Ti/Ab screening. However, the results are subjective and cannot be generalized. Further research with different scenarios and larger datasets (>5000 references) are required to support the current evidence.
Conference/Value in Health Info
2022-05, ISPOR 2022, Washington, DC, USA
Value in Health, Volume 25, Issue 6, S1 (June 2022)
Code
MSR70
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas