OBJECTIVES: To compare two methods of constructing control cohorts for a difference-in-difference (DID) econometric model in evaluating the impact on cost-saving after implementing an intervention intended to reduce variation in inpatient care for patients with community acquired pneumonia (CAP). METHODS: Using data from Office of Statewide Health Planning and Development (2000 to 2006), inpatient discharge records from four southern California licensed hospitals implementing the intervention were used to identify a CAP cohort. Controls were identified from other California county hospitals using two methods: 1) criteria-matching, which matched on hospital similarity using three criteria: total number of discharges per year, number of California Medicaid patients, and race; and 2) propensity score (PS) matching, which matched on patient baseline demographic and clinical characteristics. A DID econometric model was applied to determine intervention impact on inpatient length of stay (LOS). A zero-truncated negative binomial regression model was performed to capture the change in total LOS. The model was repeated for both criteria-matched and PS matched cohorts. Proxy cost per inpatient day ($2,127) was multiplied by the change in LOS to calculate total cost savings. RESULTS: For criteria-matched (N=36,018), mean LOS decreased for both case (0.22 days) and controls (0.10 days) following intervention implementation. DID results showed a decrease of 0.94 days per case, corresponding to $440 savings per case per year (p=0.02). Results for the PS-matched cohorts (N=22,570) also indicated a decrease in mean LOS for case (0.23 days) and controls (0.18 days) following implementation. However, DID results revealed that these differences in LOS change were not significant (p=0.83). CONCLUSIONS: The use of different methods for constructing control cohorts in the DID analysis significantly impacted the evaluation of intervention effectiveness. A PS-matched method allowed us to control observable patient characteristics and conduct a more appropriate evaluation.