A Non-Systematic Literature Review of Literature Reviews: Evaluation of Methods Implemented and Language Used to Describe Systematic and Non-Systematic Literature Reviews

OBJECTIVES: Systematic literature reviews (SLRs) are a widely used research tool with established best practices. Non-systematic literature reviews (nSLRs) encompass a range of review types and implement varying approaches to searching, screening, and data extraction. We sought to collate established guidance for SLRs/nSLRs and to understand whether there was consistency in the language and methods used to describe and conduct published SLRs/nSLRs.

METHODS: MEDLINE (via PubMed) was searched (5 Dec 2023) to identify SLRs/nSLRs published on any topic; records were restricted to those published in English, within the prior week. Title/abstract (TIAB) screening was conducted by a single researcher; includes were tagged as either SLR or nSLR. A random sample of TIAB-level includes (100 SLRs, 100 nSLRs) underwent full-text screening, involving high-level data extraction and conducted by two independent researchers. Separate searches of established evidence synthesis organizations (i.e. Cochrane, University of York, PRISMA), a selection of universities, and other research organizations were also conducted to collate recommendations or established guidance for conducting SLRs and nSLRs.

RESULTS: 1,233/1,271 TIABs identified from MEDLINE met the inclusion criteria. Of the 100 reviews defined by authors as SLRs, 93 followed established guidance; 7 did not clearly report whether established guidance for conducting SLRs was followed. Of the 100 nSLRs, authors defined their research using varying terms: review (n=55), scoping review (n=19), literature review (n=11), narrative review (n=10), comprehensive review (n=2), rapid review (n=1), structured integrative review (n=1), and tutorial review (n=1). Upon review of methods implemented in the nSLRs, 12 utilized methods sufficient for SLR definition; the remaining either leveraged elements of SLR methods or did not clearly report these details.

CONCLUSIONS: There is inconsistency in language used to describe both SLRs and nSLRs, with considerable variability seen in nSLRs. Publications should endeavour to ensure guidance for reporting SLRs are met; nSLR subtypes would benefit from improved description.