COMPARISON OF THREE META-MODELS FOR UNCERTAINTY ANALYSIS
Author(s)
Lieuw On MML1, Heeg BMS2, De Charro F2, van Hout BA31Pharmerit, Rotterdam, Zuid-Holland, Netherlands, 2Pharmerit Europe, Rotterdam, Netherlands, 3Pharmerit Ltd, York, North Yorkshire, United Kingdom
Meta-models could reduce simulation time when running probabilistic sensitivity analyses (PSA) in complex cost-effectiveness analyses models. OBJECTIVES: To compare approximations of PSA outcomes by Ordinary Least Squares (OLS), Spatial Interpolation (SI) and Gaussian Process (GP) in terms of accuracy and computation time using a simple example. METHODS: Three meta-models are used to fit the relationship between inputs and outputs considering a cost-effectiveness model addressing cardiovascular treatment and using a selection of well chosen combinations of inputs. Using separate models for both incremental costs and incremental effects, and varying the number of design-points, accuracy is measured by comparing the Root Mean Squared Error (RMSE), as comparing thousand out-of-sample predictions of the meta-models with the corresponding outputs of the cost-effectiveness model. Computation time was defined as programming and running time. The Gaussian Process emulator is used in combination with regression. RESULTS: The PSA results of the cost-effectiveness model were not linear (RESET test) in both costs and effects but the linear model showed relatively high R-squares (0.7 and 0.85). Based on RMSE, the GP gives the best results, followed closely by SI. OLS has the smallest computation time, followed by GP and SI. Latter difference mostly explained by difference in programming time. The fewer design points for the meta-models, the smaller the gap between OLS and the interpolation-based models. CONCLUSIONS: GP/SI had best accuracy but needed most computation time, while OLS is quickest but the least accurate. The difference in accuracy between SI and GP is explained by the non-linearity of the relationship. The superiority of GP over SI decreases with increasing numbers of design points.
Conference/Value in Health Info
2009-10, ISPOR Europe 2009, Paris, France
Value in Health, Vol. 12, No. 7 (October 2009)
Code
MO12
Topic
Methodological & Statistical Research
Topic Subcategory
Modeling and simulation
Disease
Cardiovascular Disorders, Multiple Diseases, Respiratory-Related Disorders