Presentation (CTI)

OBJECTIVES: To critique the application of agent based and system dynamics modelling in economic evaluations of complex, non-medicines and non-surgical interventions for health and wellbeing with a focus on quality of model reporting, economic evaluation reporting, and methods for reproducing health and health-related behaviours in models.
METHODS: Thirty-four healthcare, economics, and social science databases were systematically reviewed using pre-defined terms with no date limits. Data extraction and evaluation of reporting was via the CHEERS 2022 28-point checklist, the 2023 ISPOR Task Force Guidance on the use of complex systems models for economic evaluations of public health interventions, and a qualitative review of parameters and model logic for health and related behaviours.
RESULTS: From a search yield of 543 articles, 18 met criteria. 14 were published within the last 10yrs. Interventions included mandatory physical education in schools, commuter cycling, and COVID-19 contract tracing. Agent-based modelling was more frequently applied than system dynamics, with several instances of sequential hybrid models incorporating non-complex methods. No study provided sufficient evidence to meet all reporting standards. Few studies that performed well in CHEERS standards reporting also conformed to ISPOR recommendations and vice versa. Most descriptions of model logic and parameters spanned several publications making them challenging to collate for critique, learning, and appraisal of generalisability. Almost all studies reported uncertainty analysis, probabilistic sensitivity analysis, and two-way sensitivity analyses of key model parameters. Few described global sensitivity analysis, stakeholder involvement, or heterogeneity.
CONCLUSIONS: The use of complexity science methods for economic evaluations for complex, non-medicine interventions is increasing but reporting quality and model transparency are modest. Future evaluations would benefit from standardised formatting and accessible repositories of model documentation to support efficient appraisal of model logic and parameters. This would help advance the method and support decision-maker appraisal of the usefulness of models and model findings.