Artificial Intelligence in Evidence Synthesis: A Systematic Review and Meta-Analysis of Emerging Biologics for Improving Skeletal Health in Osteogenesis Imperfecta
Author(s)
Chengfei Li, MPH1, Zonglin Dai, PhD2, Wing Chung Tang, PhD3, Zesen Gao, MSC4, Vivien K Y Chan, PhD1, Mariana Ramirez-Posada, PhD5, Jiyeong Kim, PhD5, CL CHEUNG, PhD1, Ian Wong, PhD1, Dong Dong, PhD6, Michael To, PhD7, Dawn Craig, PhD8, Xue Li, PhD2.
1Department of Pharmacology and Pharmacy, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, Hong Kong, 2Department of Medicine, School of Clinical Medicine, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, Hong Kong, 3Yu Chun Keung Medical Library, The University of Hong Kong, Hong Kong SAR, Hong Kong, 4The University of Hong Kong Libraries, The University of Hong Kong, Hong Kong SAR, Hong Kong, 5Stanford Center for Digital Health, Department of Medicine, Stanford University, Stanford, CA, USA, 6The Jockey Club School of Public Health and Primary Care (JCSPHPC), The Chinese University of Hong Kong, Hong Kong SAR, Hong Kong, 7Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong SAR, Hong Kong, 8Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom.
1Department of Pharmacology and Pharmacy, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, Hong Kong, 2Department of Medicine, School of Clinical Medicine, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, Hong Kong, 3Yu Chun Keung Medical Library, The University of Hong Kong, Hong Kong SAR, Hong Kong, 4The University of Hong Kong Libraries, The University of Hong Kong, Hong Kong SAR, Hong Kong, 5Stanford Center for Digital Health, Department of Medicine, Stanford University, Stanford, CA, USA, 6The Jockey Club School of Public Health and Primary Care (JCSPHPC), The Chinese University of Hong Kong, Hong Kong SAR, Hong Kong, 7Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong SAR, Hong Kong, 8Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom.
OBJECTIVES: We aimed to systematically evaluate the effectiveness and safety of biologics in patients with Osteogenesis imperfecta (OI), integrating Artificial Intelligence (AI)-assisted assessment to enhance the rigor and efficiency of evidence synthesis.
METHODS: We conducted a systematic review and meta-analysis of trials assessing denosumab, setrusumab, teriparatide, and fresolimumab. Primary outcomes included changes in areal bone mineral density (aBMD) and fracture incidence. GPT-4o-2024-08-06 (API) was integrated into the workflow to support screening and quality assessment using rule-based prompts. Its performance was validated against human reviewers using sensitivity, specificity, and weighted Cohen’s kappa.
RESULTS: Eleven trials were included for a systematic review, five of which contributed to meta-analysis. Denosumab and setrusumab significantly improved lumbar spine aBMD, with denosumab achieving a 25% increase at 12 months in pediatric patients (95% CI: 18-32%), and setrusumab showing up to 16.1% improvement at 6 months. Teriparatide and fresolimumab yielded moderate aBMD gains in adults. However, no biologic significantly reduced fracture incidence compared to bisphosphonates. Safety profiles also differed: denosumab was associated with a high risk of hypercalcemia in children, whereas setrusumab had no treatment-related serious adverse events. AI achieved high sensitivity in abstract screening (96.8%) and full-text screening (90.9%), reduced total screening time by over 95% and was approximately 100 times faster per article than humans. It also showed substantial agreement with humans in quality assessment (Cohen’s kappa = 0.725), though it exhibited optimism and positional biases due to reliance on probabilistic language patterns rather than structured clinical reasoning.
CONCLUSIONS: Denosumab and setrusumab demonstrate promising efficacy in improving lumbar spine aBMD in OI, though current evidence does not support a definitive benefit of biologics in reducing fracture risk. GPT-4o supports evidence synthesis by improving screening efficiency and quality assessment, offering a scalable solution to reduce human workload. However, human oversight remains essential for tasks requiring contextual understanding and clinical reasoning.
METHODS: We conducted a systematic review and meta-analysis of trials assessing denosumab, setrusumab, teriparatide, and fresolimumab. Primary outcomes included changes in areal bone mineral density (aBMD) and fracture incidence. GPT-4o-2024-08-06 (API) was integrated into the workflow to support screening and quality assessment using rule-based prompts. Its performance was validated against human reviewers using sensitivity, specificity, and weighted Cohen’s kappa.
RESULTS: Eleven trials were included for a systematic review, five of which contributed to meta-analysis. Denosumab and setrusumab significantly improved lumbar spine aBMD, with denosumab achieving a 25% increase at 12 months in pediatric patients (95% CI: 18-32%), and setrusumab showing up to 16.1% improvement at 6 months. Teriparatide and fresolimumab yielded moderate aBMD gains in adults. However, no biologic significantly reduced fracture incidence compared to bisphosphonates. Safety profiles also differed: denosumab was associated with a high risk of hypercalcemia in children, whereas setrusumab had no treatment-related serious adverse events. AI achieved high sensitivity in abstract screening (96.8%) and full-text screening (90.9%), reduced total screening time by over 95% and was approximately 100 times faster per article than humans. It also showed substantial agreement with humans in quality assessment (Cohen’s kappa = 0.725), though it exhibited optimism and positional biases due to reliance on probabilistic language patterns rather than structured clinical reasoning.
CONCLUSIONS: Denosumab and setrusumab demonstrate promising efficacy in improving lumbar spine aBMD in OI, though current evidence does not support a definitive benefit of biologics in reducing fracture risk. GPT-4o supports evidence synthesis by improving screening efficiency and quality assessment, offering a scalable solution to reduce human workload. However, human oversight remains essential for tasks requiring contextual understanding and clinical reasoning.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
EPH19
Topic
Clinical Outcomes, Epidemiology & Public Health, Methodological & Statistical Research
Topic Subcategory
Safety & Pharmacoepidemiology
Disease
Musculoskeletal Disorders (Arthritis, Bone Disorders, Osteoporosis, Other Musculoskeletal), No Additional Disease & Conditions/Specialized Treatment Areas, Rare & Orphan Diseases