Presentation (CTI)

OBJECTIVES: While the choice of platform used to construct cost-effectiveness analysis (CEA) models should not be as important as ensuring these models can produce valid results, Microsoft Excel has been the preferred software for CEAs for Health Technology Assessment (HTA) submissions. Validation of R-based models may rely on comparison of numerical results against the existing Excel-based models. We conducted a scoping review to examine the historical performance of Excel’s statistical computing and numerical accuracy relative to R and contextualize the impact of using Excel-models to validate R-based models.
METHODS: Following the JBI Scoping Review Methodology Guidance, we conducted a scoping review to identify previous evaluation of Excel’s statistical computing and numerical accuracies. We searched peer-reviewed publications using the terms “Microsoft Excel” AND “validation” AND (numerical accuracy OR statistical performance) on PubMed complemented with hand-searches.
RESULTS: Our search yield eight peer-reviewed publications that evaluated the performance on basic probabilistic and statistical computations of six different versions from Excel 97 to Excel 2010. These studies have shown Excel’s inadequate performance as a statistical tool. Some evaluation showed that earlier versions produced zero significant digits of accuracy when computing probabilities or quantiles from standard distributions, alongside inaccuracies in basic statistical tests. Excel also truncates any numerical values beyond the 15th significant digit to zero, which is a different behaviour from standard numerical computing practices. This can therefore introduce systematic discrepancies to R.
CONCLUSIONS: As the industry moves towards open-source software like R, practitioners may attempt to validate the results outputted by R in Excel to ensure consistency. We note that modern versions of Excel have not been validated or tested to the same extent that previous versions have where this testing scrutiny is commonly applied to R. Our findings suggest that Excel should not be used to validate the R-based CEAs.