Deploy-AI: A Framework and Future Checklist for Facilitating Deployment of Clinical Prediction Models

OBJECTIVES: The scientific literature contains clinical prediction models for numerous health outcomes but lacks guidelines on how best to design algorithms with downstream deployment considerations in mind, such as electronic health record integrated clinical decision support tools (CDSTs) to support healthcare professionals (HCPs). We propose Deploy-AI, a framework to guide up-front algorithm design to ensure that models are fit-for-purpose downstream when deployed in a clinical setting. Without such considerations, models may be suboptimal and even unusable. For instance, a CDST developed for a specialist setting could be entirely ineffective in primary care where available data, technical infrastructure and clinical end users may be vastly different.

METHODS: A checklist of deployment considerations was agreed by the authors, based on extensive experience in clinical practice and health informatics. The checklist comprises multiple items across three domains: clinical, technical, and regulatory.

RESULTS: Clinical considerations include understanding the clinical workflow, the role of the CDST to closing a specific care gap(s), the role of the end user and the type of model outputs required for the HCP to make an informed decision. The technical domain includes assessing data availability at the point of model execution, the generalisability of an existing algorithm (if applicable) for deployment in the target environment and assessing the operational feasibility of the intended technical workflow. Regulatory considerations include information governance frameworks, data privacy frameworks, life science industry code of conduct guidelines and software as a medical device factors.

CONCLUSIONS: Deploy-AI can guide clinical AI algorithm developers to design algorithms to best meet the downstream requirements of a CDST. This complements other frameworks, such as TRIPOD+AI, which focuses on reporting. We intend to illustrate an application of the points presented in this checklist in a future study of early axial spondyloarthritis diagnosis.