Objective: Difference-in-difference (DID) and propensity score matching (PSM) methods are two ways of evaluating the effect of changes in policies influencing drug prescribing or medications.  The objective of this study is to compare and contrast the strengths and weakness and different types of information provided by the DID and PSM methods in the context of determining the impact of an insurance plan restriction on binary health care utilization outcomes. Data: Adult patients diagnosed with diagnosis A were selected from MarketScan® Commercial Database (2005-2007), a large claims data form persons with employ-based insurance. Methodology:  The DID analysis was implemented by a GEE regression with logit link, binomial variance function, and exchangeable correlation structure on panel data with pre- and post-restriction period. The dependent variables included inpatient hospitalization, ER visit, use of alternative drug B, and use of alternative drug C. The key independent variables included an indicator for plan restriction, an indicator for pre- or post- restriction period, and an interaction term between them. Demographic and clinical characteristics were included as covariates. The odds ratios (OR) of the interaction terms were considered as the DID estimates. In the PSM method, patients in restricted and unrestricted plans were propensity score matched based on pre-implementation characteristics, followed by logistic regressions. Results: Without controlling for any covariate, the restriction effects were non-significant on all outcomes using DID analysis. The restriction effects were significant for ER visit and hospitalization using PSM. When all characteristics were controlled, the restriction effects were non-significant in both methods, except the outcome of drug B use. Conclusion: DID and PSM- based methods may produce different estimates. Neither method can fully control all potential confounders. Compared with the PSM method, DID method uses same individual as the control, but is restricted to studies when pre- and post-event data are available.