Presentation (CTI)

OBJECTIVES: Identifying patient-reported outcome (PRO) instruments from the literature can be time-consuming and resource intensive. Typically PRO instruments are extracted directly from the abstracts, with full text papers only being sought if insufficient detail is included. AI tools have the potential to enhance efficiency while maintaining accuracy. This study compared the strengths and limitations of two AI-assisted platforms, as well as traditional methods, for screening abstracts to identify PRO instruments.
METHODS: A targeted literature review to identify PRO instruments for head and neck squamous cell carcinoma was completed. Manual screening involved one primary reviewer screening all abstracts with 10% double-screening. AI-assisted screening was performed in Covidence and Rayyan. Abstract screening time and platform-specific strengths and limitations were recorded.
RESULTS: Manual screening required 11.5 hours, Covidence 3.75 hours, and Rayyan 3.75 hours. Both AI-assisted platforms had user-friendly interfaces with keyword highlighting and relevance-ranking features. Covidence offered conflict resolution capabilities and team collaboration but was limited to reviews with >100 studies and lacked exclusion documentation. Rayyan enabled addition of reviewer notes, but had limitations in conflict resolution, requiring individual vote changes. Manual screening allowed clear publication type identification and detailed note-taking. Both AI-assisted platforms identified more studies for inclusion than manual screening.
CONCLUSIONS: AI-assisted screening was faster than traditional methods, but some limitations and considerations for implementation were identified. Both AI-assisted platforms identified more studies for inclusion, due to difficulty distinguishing conference abstracts from manuscripts, requiring full-text screening. AI platform selection should be tailored to the requirements of the literature review, considering factors such as review size, collaboration needs and type of publication. Limitations of this study included the use of free AI platform versions and potential researcher learning effects. Future research should explore the use of AI-assisted platforms for other aspects of literature reviews, including a comparison of accuracy across methods.