Qualitative Analysis of Results From a Delphi Study to Formalize a Treatment Protocol for Children With Early Diagnosed Duchenne Muscular Disease (DMD) Using Giles®, an Artificial Intelligence Agent for Healthcare Research
Author(s)
Michelle Lorentzos, MD1, Laurent Servais, MD, PhD2, Julie Parsons, MD3, Kristi Jones, PhD4, Richard Shell, MD5, Stefan Spinty, MD6, Richard Finkel, MD7, Molly Colvin, PhD8, Michael Simpson, .9, Diane Murrell, MSW10, Gina O’Grady, MD11, Meeta Cardon, MD12, Thomas Sejersen, MD, PhD13, Damjan Osredkar, MD, PhD14, Maggie Walter, MD15, Anne Stratton, MD16, Lisa Tang, MSc17, Rachel Salmon, MSc18, Dino Masic, PhD19, Annie Poll, PhD20, Iftekhar Khan, PhD21, Ralph Crott, MPH, MSc, PhD22, Rabiah Begum, MSc22, Kiran Dhaliwal, BSc23, Constantin Gorgan, MBA23, Kamran Khan, MSc22.
1Sydney Children’s Hospitals Network, Sydney, Australia, 2University of Oxford, Oxford, United Kingdom, 3University of Colorado, Aurora, CO, USA, 4Sydney Children's Hospitals Network, Sydney, Australia, 5Nationwide Children’s Hospital, Columbus, OH, USA, 6Alder Hey Children’s Hospital, Liverpool, United Kingdom, 7St. Jude Children's Research Hospital, Memphis, TN, USA, 8Massachusetts General Hospital, Boston, MA, USA, 9Duchenne Australia, Perth, Australia, 10Texas Children’s, Houston, TX, USA, 11Starship Children’s Health, Te Whatu Ora Health New Zealand, Auckland, New Zealand, 12University of New Mexico, Albuquerque, NM, USA, 13Karolinska Institutet, Stockholm, Sweden, 14University Children's Hospital, Ljubljana, Slovenia, 15LMU University Hospital, Munich, Germany, 16Children’s Hospital Colorado, Aurora, CO, USA, 17Children's Hospital at Westmead, Sydney, Australia, 18The Llandough Centre for Spinal and Neurological Rehabilitation, Llandough, United Kingdom, 19TREAT-NMD® Services Ltd, Newcastle Upon Tyne, United Kingdom, 20TREAT-NMD Services Ltd, Newcastle Upon Tyne, United Kingdom, 21University of Warwick, Coventry, United Kingdom, 22Regulatory Scientific and Health Solutions, Solihull, United Kingdom, 23Giles AI, London, United Kingdom.
1Sydney Children’s Hospitals Network, Sydney, Australia, 2University of Oxford, Oxford, United Kingdom, 3University of Colorado, Aurora, CO, USA, 4Sydney Children's Hospitals Network, Sydney, Australia, 5Nationwide Children’s Hospital, Columbus, OH, USA, 6Alder Hey Children’s Hospital, Liverpool, United Kingdom, 7St. Jude Children's Research Hospital, Memphis, TN, USA, 8Massachusetts General Hospital, Boston, MA, USA, 9Duchenne Australia, Perth, Australia, 10Texas Children’s, Houston, TX, USA, 11Starship Children’s Health, Te Whatu Ora Health New Zealand, Auckland, New Zealand, 12University of New Mexico, Albuquerque, NM, USA, 13Karolinska Institutet, Stockholm, Sweden, 14University Children's Hospital, Ljubljana, Slovenia, 15LMU University Hospital, Munich, Germany, 16Children’s Hospital Colorado, Aurora, CO, USA, 17Children's Hospital at Westmead, Sydney, Australia, 18The Llandough Centre for Spinal and Neurological Rehabilitation, Llandough, United Kingdom, 19TREAT-NMD® Services Ltd, Newcastle Upon Tyne, United Kingdom, 20TREAT-NMD Services Ltd, Newcastle Upon Tyne, United Kingdom, 21University of Warwick, Coventry, United Kingdom, 22Regulatory Scientific and Health Solutions, Solihull, United Kingdom, 23Giles AI, London, United Kingdom.
OBJECTIVES: Increasing prevalence of new-born genetic screening has led to earlier diagnoses of Duchenne Muscular Dystrophy (DMD). A treatment framework was published to provide clinicians with guidance on best practices in care and treatment for newly diagnosed DMD patients and their families. TREAT-NMD assembled a panel of experts in the DMD field to review the framework and a Delphi study was used to determine the level of consensus.
METHODS: Fourteen panel members scored each action item of the treatment framework via anonymised online surveys using a seven-point Likert scale (-3 (strong disagreement) to +3 (strong agreement)), providing explanations for their ratings. Four criteria were defined for consensus: (i). Median score of ≥2, (ii). Interquartile Range (IQR) ≤ 1, (iii) Median score of >2, (iv). Minimum score ≥ 0. Qualitative analyses of panellist plain text comments on their responses across two survey rounds were undertaken by giles®, an Artificial Intelligence (AI) agent using a large language model (LLM).
RESULTS: 12 out of 35 action items did not reach consensus in round 1and were re-evaluated in round 2. Cronbach’s Alpha was reported at 0.861 and 0.802, for responses to action items in rounds 1 and 2, respectively, indicating ‘good’ internal consistency of the data from both rounds. The qualitative analyses included a summary of explanations provided by panellists for their ratings of the action items. 43% to 86% of panellists provided explanations for their ratings in round 2 and giles® was able to highlight where panellists aligned on reasonings for their scores.
CONCLUSIONS: This Delphi study provided clarity and confidence around several central pillars of the early diagnosis framework. The giles® tool was able to successfully identify where panellists were aligned on each of the action items and where they differed in opinion by providing a short and consistent summary of all responses.
METHODS: Fourteen panel members scored each action item of the treatment framework via anonymised online surveys using a seven-point Likert scale (-3 (strong disagreement) to +3 (strong agreement)), providing explanations for their ratings. Four criteria were defined for consensus: (i). Median score of ≥2, (ii). Interquartile Range (IQR) ≤ 1, (iii) Median score of >2, (iv). Minimum score ≥ 0. Qualitative analyses of panellist plain text comments on their responses across two survey rounds were undertaken by giles®, an Artificial Intelligence (AI) agent using a large language model (LLM).
RESULTS: 12 out of 35 action items did not reach consensus in round 1and were re-evaluated in round 2. Cronbach’s Alpha was reported at 0.861 and 0.802, for responses to action items in rounds 1 and 2, respectively, indicating ‘good’ internal consistency of the data from both rounds. The qualitative analyses included a summary of explanations provided by panellists for their ratings of the action items. 43% to 86% of panellists provided explanations for their ratings in round 2 and giles® was able to highlight where panellists aligned on reasonings for their scores.
CONCLUSIONS: This Delphi study provided clarity and confidence around several central pillars of the early diagnosis framework. The giles® tool was able to successfully identify where panellists were aligned on each of the action items and where they differed in opinion by providing a short and consistent summary of all responses.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
MSR177
Topic
Health Service Delivery & Process of Care, Methodological & Statistical Research, Real World Data & Information Systems
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Survey Methods
Disease
Musculoskeletal Disorders (Arthritis, Bone Disorders, Osteoporosis, Other Musculoskeletal)