OBJECTIVES: Transcranial magnetic stimulation (TMS) is FDA cleared for use in pharmacoresistant depression.  Two sham-controlled trials have confirmed its efficacy and safety. However, TMS has not been directly compared to pharmacotherapy. Propensity score matching was used to compare the effectiveness of TMS to pharmacotherapy. Prospectively collected data from a pragmatic study of 305 patients treated in routine practice with TMS were matched to patients from the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study. METHODS: TMS patients were propensity-score matched to STAR*D patients on baseline characteristics using a 1:1 greedy matching algorithm. An unequal drug resistance distribution in the two populations allowed only 222 patients to match well on the first attempt. A subsequent re-matching of the remaining TMS subjects to the full STAR*D control population was performed to produce a complete match.  This “double-dipping” approach enabled a successful complete match for all 305 TMS patients. RESULTS: The matched STAR*D and TMS populations were similar at baseline. QIDS-SR outcomes at 6 weeks showed that the TMS group had a greater clinical improvement (P<0.0001). At 6-weeks 53% of TMS patients had no or mild depression versus 38% for STAR*D (p=0.0023). Sensitivity analysis was used to estimate the potential effects of any remaining selection biasing factors, and confirmed an unlikely impact on results. CONCLUSIONS: The varying distribution of the severity of baseline treatment resistance between the TMS and STAR*D populations made it impossible to achieve a complete match in the first matching attempt. Subsequent, “double-dipping” allowed tight matching on baseline variables. We accepted the risk to internal validity posed by the remaining selection bias or confounding and the small impact to variability due to non-independence, in exchange for gaining an increased external validity for this difficult to match group. Matching hard-to-match groups requires a trade-off between risks to internal and external validity.