Time to Stop Sinking Resources Into Developing Predictive Models That Yield No Value: A Framework and Opportunities for Health Economic Methods Development to Reduce Waste in Prediction Model Development, Including Use of AI

OBJECTIVES: Unprecedented resources are being invested (or perhaps sunk is more apt) into the proliferation of thousands of new predictive models for healthcare contexts each year. Few progress past development stage, are generalisable, or demonstrate any value (as a concept distinct from statistical performance) for consumers or healthcare systems when operationalised. This conceptual work describes methodological opportunities for considering value throughout the prediction model development pipeline building off existing frameworks from adjacent fields and highlighting further opportunities for health economic methodological development.

METHODS: A preliminary framework was developed aligning opportunities for methods development exploring likely real-world value (for consumers or healthcare systems) across the model development pipeline. This framework draws on a multi-year program of systematic and scoping reviews, qualitative, mixed methods and quantitative studies, including model development and optimisation studies, in addition to implementation-evaluation of clinical decision support systems and policy analysis.

RESULTS: The preliminary framework includes opportunities for health economic methods advancement across stages of i) problem definition and quantification, ii) model development, validation and optimisation; iii) operationalisation and integration into decision support systems, and iv) implementation, evaluation, iteration and sustainment. Exemplar opportunities include: i) estimating the extent to which the target problem(s), which are often complex and multifaceted, would be resolved by prediction; ii) whether prediction targets are optimised to lead to high-value care (versus predicting events unlikely to be modified); iii) quantifying downstream consequences of prediction (including responding to false positives); iv) value-add of new prediction-based decision support (versus otherwise expected actions).

CONCLUSIONS: Prediction model development, including use of artificial intelligence, that will not improve outcomes for consumers or the healthcare system wastes scarce clinical and technical resources. There is considerable opportunity to advance methodological work to embed early and continued consideration of likely real-world value in predictive model development decision points to reduce this waste.