The Who, When, and How of Model Validation: A Targeted Literature Review of Roles and Opportunities for Improving Credibility Through All Phases of Model Development

OBJECTIVES: Health economic (HE) decision-analytic models are a critical component of health technology assessments (HTAs) and evaluation of model validity can support a decision-maker’s ability to trust model results. While most models undergo some level of validation, recommendations around how, when, and by whom it should be conducted vary widely. This study assesses current literature and summarizes the guidance on HE model validation.

METHODS: A targeted literature review was conducted via an electronic search in PubMed followed by citation searching of key papers. Additionally, model validation documents published by national HTA bodies were considered. The PubMed search included HE modelling terminology alongside validation terms. Exclusion criteria included non-HE model types and missing discussion of validation methodology.

RESULTS: A total of 322 studies were identified following both the PubMed and citation searches. After title and abstract screening and full-text review, eight articles were identified from the PubMed search and eight from citation searches. Six documents from HTA body websites were also included. The 22 sources covered 21 unique approaches. Of these, over half (12/21) provided conceptual frameworks for assessing quality, and the remainder (9/21) provided specific tools for assessing the validity of HE models. Most approaches (15/21) reported methods which required validation by a modeller with technical expertise, six (6/21) provided guidance for decision-makers or model users, and two (2/21) described the role of clinical experts in validation. Most approaches recommended verification of technical methods (15/21), alongside assessment of model concept and face validity (20/21). Validation was most frequently recommended during model finalization (10/21) and after model completion (15/21).

CONCLUSIONS: Current literature provides a range of theoretical frameworks and assistive tools to guide the critical assessment of HE model validity. This study can help modellers, assessors, and model users better understand role-specific validation recommendations across different stages of model development.