OBJECTIVES:

Health economic (HE) models are routinely developed and used to support health policy decisions, but are often not publicly available. Additionally, relatively few HE models are extensively validated and validation efforts are only occasionally reported systematically. This lack of validation may undermine HE model credibility and increases the chance of taking wrong decisions, potentially leading to health losses. Solving this issue requires new approaches to validating HE models when the underlying model is unavailable. Metamodelling, i.e. fitting a statistical model (e.g. linear regression) to a HE model's inputs and corresponding outputs, can generate insights in how such models work and into their inputs-outputs relationships.

The aim of this study was to develop an interactive dashboard to systematically explore and validate HE models’ inputs and outputs.

METHODS:

The R shiny Probabilistic Analysis Check dashBOARD (PACBOARD) was developed using insights from literature, health economists, and a data scientist. PACBOARD requires users to upload the inputs and corresponding model outputs of a probabilistic analysis.

Functionalities of PACBOARD are: 1) validating and inspecting model inputs and outputs using standardised validity tests (e.g. checking whether all cost inputs are positive) and interactive plots; 2) visualising and investigating the inputs-outputs relationships using metamodelling. PACBOARD also allows to make predictions using the fitted metamodel.

A HE model containing errors (e.g. negative costs and transition probabilities) was developed to test the functionalities of PACBOARD. PACBOARD metamodelling predictions were validated against the original model’s outputs.

RESULTS:

PACBOARD automatically identified all errors in the incorrect HE model. Metamodelling predictions were similar compared to the original model outputs.

CONCLUSIONS:

PACBOARD is a unique tool to standardise and increase the transparency of HE model validation efforts, without requiring access to the original HE model. It increases the feasibility of validating HE models regardless of model type and disease domain.