UNCOVERING THE GAP BETWEEN MODEL VALIDATION RECOMMENDATIONS AND PRACTICE
Author(s)
Kleintjens J*, Akpo H Deloitte, Diegem, Belgium
OBJECTIVES: Health technology agencies and payers rely on economic models to evaluate the value of new treatments. It is therefore critical to ensure the scientific credibility of these models through validation, which is also required by NICE since 2012. The aim of this study is to understand which of the validation methods recommended in health economics and financial guidelines are practically used across different disease areas and business fields. METHODS: For this study we reviewed guidelines, economic models for diabetes, cardiovascular diseases, and cancers, as well as risk-prediction models. The identified validation methods were collected and categorized into three groups: face validity, internal verification, and cross validity. RESULTS: Twenty-two (22) economic models, 8 risk prediction models, 5 financial and 7 health economics guidelines were identified. In the guidelines we found 3, 14 and 5 methods within the face validity, internal verification, and external validity categories respectively. Almost three quarters of the 22 methods identified were also described in the ISPOR guideline on model validation and the other guidelines described 5 to 15 methods. External validation was done for 14 out of the 22 economic models reviewed in which (part of) the model results were compared against other clinical/epidemiology studies (14 models) or other validated models (10 models). Internal verification was mainly done through comparison against the trials that the models were built on (10 models). Out of the remaining 13 internal verification methods only 3 were performed for the economic models. Face validation was only performed for 3 economic models. CONCLUSIONS: This study found that there is a substantial gap between the validation steps recommended and those actually performed. Hence a more consistent and pragmatic approach needs to be taken to validate economic models. In addition, the validation methods performed need to be consistently reported.
Conference/Value in Health Info
2013-11, ISPOR Europe 2013, The Convention Centre Dublin
Value in Health, Vol. 16, No. 7 (November 2013)
Code
PRM38
Topic
Economic Evaluation
Topic Subcategory
Cost/Cost of Illness/Resource Use Studies
Disease
Cardiovascular Disorders, Diabetes/Endocrine/Metabolic Disorders, Oncology, Respiratory-Related Disorders