METHODOLOGY FOR ESTABLISHING INTERNAL AND EXTERNAL VALIDITY WHEN PROPENSITY SCORE MATCHING IS USED

Author(s)

Eisenberg D*1;Wasser T1;Placzek H2, Luthra R1 1HealthCore, Inc., Wilmington, DE, USA, 2HealthCore, Inc., Andover, MA, USA

OBJECTIVES: Propensity score matching (PSM) is an approach commonly used when treatment and control groups are thought to be different on key study variables. When the control group is larger than the treatment group, (as large as 20:1) a good match might be easy to obtain. However, differences may exist between the matched controls and the unmatched controls, indicating poor generalizability of study results. METHODS: Groups for the analysis are the unmatched controls (UM), the matched controls (MC) and the treatment cohort (TRT). Analysis methods for these groups in a fully crossed method and interpretation of the results will determine internal (IV) and external validity (EV). Analysis comparing the groups against the outcomes variable will determine if variables need to be controlled for in models that may be developed. RESULTS: After the PSM is conducted MC and TRT groups should be compared on the matched variables.  Differences at this stage would indicate a poor match and a low level of IV. MC and UM should also be compared on the variables used for matching, as well as the outcome variables of interest. Significant differences on the matched variables would indicate low EV and poor generalizability of results, while differences of MC and UM groups and UM and TRT groups on the outcome variables would indicate that statistical models would need to address covariates as potential confounding effects would be present. Analysis methods can be fit statistics (chi-square or equivalence tests) or typical inferential methods with adjusted p-values greater than 0.05. CONCLUSIONS: It is important that research studies maintain good IV and EV. This is often complicated in research where the controls vastly outnumber the treatment group. Proper statistical analysis can go a long way to test and clarify data to make the results as meaningful as possible.

Conference/Value in Health Info

2013-11, ISPOR Europe 2013, The Convention Centre Dublin

Value in Health, Vol. 16, No. 7 (November 2013)

Code

PRM206

Topic

Methodological & Statistical Research

Topic Subcategory

Confounding, Selection Bias Correction, Causal Inference

Disease

Multiple Diseases

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×