AN ACCESSIBLE COMPARISON OF TRADITIONAL STATISTICAL AND MACHINE LEARNING APPROACHES TO ANALYSIS OF REAL WORLD DATA- WHICH, WHEN AND WHY?
Author(s)
Merinopoulou E1, Oguz M1, Chu BC2, McDonald L3, Ulvestad M4, Cox A1, Ramagopalan S3
1Evidera, London, UK, 2Evidera, Bethesda, MD, USA, 3Bristol-Myers Squibb, Uxbridge, UK, 4Bristol-Myers Squibb, Oslo, Norway
OBJECTIVES: Machine Learning (ML) is becoming an increasingly important approach for the analysis of real-world data and there is a lively debate about which approach is best among experts. However, for non-statisticians there is a general lack of clarity around appropriate use of ML versus a traditional statistical approach (TSA). Through a practical example, we set out to examine the similarities and differences between the two approaches in an unbiased exploration. METHODS: We used both approaches to examine the possible predictors being diagnosed with Non-Valvular Atrial Fibrillation (NVAF) before or after a stroke using a large US Health Insurance Claims dataset (Pharmetrics). Analysts were blinded and asked only to address the study objective using either a TSA or a ML approach. RESULTS: Logistic regression was used for the TSA, the most important predictor of after stroke diagnosis was aortic plaque (OR=1.4[95% CI1.2~1.6]) and the TSA had an area under the curve (AUC) of 0.67. For the ML random forest was used and the most important predictor of after stroke diagnosis was a healthcare cost (mean decrease in accuracy=4.3%) with an overall AUC of 0.80. Comparing the analyses we found that machine learning approach considered more variables in the analysis, without the variable preselection steps present in the TSA. It was therefore stronger in identifying potentially new predictive relationships. Addressing data quality issues was considerably more complex for the TSA. The TSA provided a more interpretable assessment of the influence of discovered predictor variables in the form of an odds ratio, highlighting the fact that this approach is stronger in making inference. CONCLUSIONS: This work discusses the strengths and weakness of ML versus TSA and helps to clarify the similarities and potential use cases for each approach.
Conference/Value in Health Info
2017-11, ISPOR Europe 2017, Glasgow, Scotland
Value in Health, Vol. 20, No. 9 (October 2017)
Code
PRM217
Topic
Methodological & Statistical Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
Cardiovascular Disorders