Estimating the Expected Value of Perfect Parameter Information by Machine-Learning

OBJECTIVES: The expected value of perfect parameter information (EVPPI) for a decision analytic model quantifies the expected benefit of learning the values of a set of uncertain parameters. EVPPI was first estimated via a 2-level Monte Carlo procedure where the parameters of interest are sampled in an outer loop, then conditional on these, the remaining parameters are sampled in an inner loop. This procedure is very computationally cumbersome especially when the parameters of interest are correlated. To circumvent these computational difficulties several studies have proposed, using meta-models, such as Gaussian Process Regression (GPR) or Generalised Additive Models and the model's probability sensitivity analysis (PSA) results to estimate EVPPI. These methods require estimating as many regressions as decisions minus one, this can be computationally cumbersome when the number of decisions is large. This study aims to provide a new method that can handle an arbitrarily large number of decisions, in a computationally efficient way.

METHODS: In the study, we propose a novel method to calculate EVPPI that leverages the machine learning technique random forests. We do this by first reframing EVPPI as a classification problem and using random forests to predict the optimal decision (from the PSA results) conditional on the set of parameters the researcher is interested in. We then use this conditional prediction to calculate EVPPI.

RESULTS: We find in a simulated case study with three decisions, that the proposed method performs comparably to existing methods and is an order of magnitude faster than the GPR.

CONCLUSIONS: The key advantage of the proposed method over existing methods is it can handle models of any complexity including those with an arbitrarily large number of decisions. Random forest is chosen over other machine-learning-based classification procedures as the predictions from the individual trees provide a natural way to calculate standard errors.