Presentation (CTI)

OBJECTIVES: Current Cochrane systematic literature review (SLR) guidance recommends including grey literature sources (e.g., conference proceedings, trial registries, and regulatory documents) to capture unpublished data and reduce publication bias. However, grey literature searching is frequently manual, resource-intensive, and challenging to integrate into AI-driven SLR platforms used to improve efficiencies and enable rapid, frequent evidence updates (“living SLRs”). This umbrella review aimed to investigate the value of including grey literature in SLRs and to determine whether its inclusion meaningfully influences evidence synthesis outcomes.
METHODS: We conducted an umbrella review of SLRs and meta-analyses (MAs) that reported the impact of grey literature on findings including publication bias and treatment effect estimates. The review was conducted using the Nested Knowledge platform, with AI-driven search strategies, criteria-based screening, and data extraction, with human-in-the-loop oversight. Outcomes assessed included publication type, reporting quality, impact on pooled estimates, and potential for bias.
RESULTS: SLRs with evidence for the exclusion of grey literature reported poor quality and inconsistencies (e.g., in congress abstracts compared to full-texts, little impact on pooled effect estimates or overall conclusions) plus the need for intensive resources. In contrast, SLRs with evidence for inclusion tended to report publication bias and overestimation of effect size when grey literature was omitted and noted the importance of this source when evidence is scarce. Sensitivity analyses for MAs with and without grey literature were recommended.
CONCLUSIONS: These findings suggest that the value of grey literature inclusion in SLRs is context-dependent (e.g., for HTA submissions that rely on robust evidence from SLRs that need to be frequently and rapidly updated) and the inclusion of grey literature may not substantially strengthen the overall body of evidence. Deprioritising grey literature in such contexts could improve feasibility without compromising evidence integrity, to support more efficient, scalable approaches to living SLRs.