METHODOLOGY FOR ESTABLISHING INTERNAL AND EXTERNAL VALIDITY WHEN PROPENSITY SCORE MATCHING IS USED
Author(s)
Eisenberg D*1;Wasser T1;Placzek H2, Luthra R1 1HealthCore, Inc., Wilmington, DE, USA, 2HealthCore, Inc., Andover, MA, USA
OBJECTIVES: Propensity score matching (PSM) is an approach commonly used when treatment and control groups are thought to be different on key study variables. When the control group is larger than the treatment group, (as large as 20:1) a good match might be easy to obtain. However, differences may exist between the matched controls and the unmatched controls, indicating poor generalizability of study results. METHODS: Groups for the analysis are the unmatched controls (UM), the matched controls (MC) and the treatment cohort (TRT). Analysis methods for these groups in a fully crossed method and interpretation of the results will determine internal (IV) and external validity (EV). Analysis comparing the groups against the outcomes variable will determine if variables need to be controlled for in models that may be developed. RESULTS: After the PSM is conducted MC and TRT groups should be compared on the matched variables. Differences at this stage would indicate a poor match and a low level of IV. MC and UM should also be compared on the variables used for matching, as well as the outcome variables of interest. Significant differences on the matched variables would indicate low EV and poor generalizability of results, while differences of MC and UM groups and UM and TRT groups on the outcome variables would indicate that statistical models would need to address covariates as potential confounding effects would be present. Analysis methods can be fit statistics (chi-square or equivalence tests) or typical inferential methods with adjusted p-values greater than 0.05. CONCLUSIONS: It is important that research studies maintain good IV and EV. This is often complicated in research where the controls vastly outnumber the treatment group. Proper statistical analysis can go a long way to test and clarify data to make the results as meaningful as possible.
Conference/Value in Health Info
2013-11, ISPOR Europe 2013, The Convention Centre Dublin
Value in Health, Vol. 16, No. 7 (November 2013)
Code
PRM206
Topic
Methodological & Statistical Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
Multiple Diseases