Integrating Generative Artificial Intelligence Into Evidence Synthesis: Opportunities, Safeguards, and Methodological Refinements
Abstract
I read with genuine admiration the article by Reason et al, which
presents an innovative exploration of generative artificial intelligence
as an “artificial intelligence statistician” capable of autonomously
selecting appropriate models and executing network meta-analyses (NMAs).
The study is particularly compelling in demonstrating that a large
language model (LLM)-based automated process could consistently identify
the correct statistical framework across all 16 National Institute for
Health and Care Excellence TSD2 case studies, generate appropriate model
specifications, and reproduce network diagrams and executable code with
high fidelity. The finding that the artificial intelligence
(AI)-generated incremental treatment effects closely replicated
published estimates—for example, yielding d12 = −0.26 (95%
CrI −0.36, −0.16) in the fixed-effects blocker example, identical to the
National Institute for Health and Care Excellence TSD2 results—is both
technically impressive and highly encouraging for the future automation
of evidence-synthesis workflows.
Authors
Weihao Cheng Hangyu Zhu Yuanyuan Lian