COMPARISON OF THREE META-MODELS FOR UNCERTAINTY ANALYSIS

Author(s)

Lieuw On MML1, Heeg BMS2, De Charro F2, van Hout BA31Pharmerit, Rotterdam, Zuid-Holland, Netherlands, 2Pharmerit Europe, Rotterdam, Netherlands, 3Pharmerit Ltd, York, North Yorkshire, United Kingdom

Meta-models could reduce simulation time when running probabilistic sensitivity analyses (PSA) in complex cost-effectiveness analyses models.  OBJECTIVES: To compare approximations of PSA outcomes by Ordinary Least Squares (OLS), Spatial Interpolation (SI) and Gaussian Process (GP) in terms of accuracy and computation time using a simple example. METHODS: Three meta-models are used to fit the relationship between inputs and outputs considering a cost-effectiveness model addressing cardiovascular treatment and using a selection of well chosen combinations of inputs. Using separate models for both incremental costs and incremental effects, and varying the number of design-points, accuracy is measured by comparing the Root Mean Squared Error (RMSE), as comparing thousand out-of-sample predictions of the meta-models with the corresponding outputs of the cost-effectiveness model. Computation time was defined as programming and running time. The Gaussian Process emulator is used in combination with regression.  RESULTS: The PSA results of the cost-effectiveness model were not linear (RESET test) in both costs and effects but the linear model showed relatively high R-squares (0.7 and 0.85). Based on RMSE, the GP gives the best results, followed closely by SI. OLS has the smallest computation time, followed by GP and SI. Latter difference mostly explained by difference in programming time. The fewer design points for the meta-models, the smaller the gap between OLS and the interpolation-based models. CONCLUSIONS: GP/SI had best accuracy but needed most computation time, while OLS is quickest but the least accurate. The difference in accuracy between SI and GP is explained by the non-linearity of the relationship. The superiority of GP over SI decreases with increasing numbers of design points.

Conference/Value in Health Info

2009-10, ISPOR Europe 2009, Paris, France

Value in Health, Vol. 12, No. 7 (October 2009)

Code

MO12

Topic

Methodological & Statistical Research

Topic Subcategory

Modeling and simulation

Disease

Cardiovascular Disorders, Multiple Diseases, Respiratory-Related Disorders

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×