OBJECTIVES: To test a more uniformly structured approach to evaluating evidence for formulary decision making among managed care decision makers. METHODS: A structured framework assessing the impact of costs, efficacy benefits, safety concerns, and certainty of efficacy and safety evidence on formulary access was tested using four hypothetical clinical conditions (breast cancer, osteoporosis, Alzheimer’s disease, and hypertension).  After recruitment from a convenience sample, respondents were assigned via balanced incomplete block design to rate 12 scenarios per clinical condition on a 1-9 scale (1=no access, 9= open access).  Distribution of ratings, means, medians, and rate of disagreement was calculated. Mixed effects linear regression models using maximum likelihood, with rater as a random effect, were fit to estimate the association of clinical condition, cost, efficacy certainty, safety certainty, efficacy benefit, and safety concerns with level of formulary access.  RESULTS: Seventy-nine P&T decision-makers completed the survey between February-October 2011 resulting in 3783 evaluable responses (2823=pharmacy directors, 960=medical directors). Mean ratings were lower among pharmacy directors vs. medical directors (3.92 vs. 4.36, p=0.093).  Adjusted formulary access ratings differed by clinical condition (breast cancer 4.40; osteoporosis 4.04; Alzheimer’s 4.00, and hypertension 3.70).  Individual raters had substantial disagreement in individual scenario ratings (MAD 0.25-2.19).  Across all conditions, greater formulary access was significantly associated with greater certainty of safety evidence (high vs. low 1.60), greater certainty of efficacy evidence (0.72), greater magnitude of efficacy benefit (0.43), fewer safety concerns (0.22) and lower comparative cost (0.54). CONCLUSIONS: Greater formulary access was associated with greater efficacy benefit, certainty of efficacy and safety evidence, lower costs, and fewer safety concerns.  Despite substantial inter-rater variation within each scenario, the structure of the framework held during testing with a broad number of P&T decision-makers, suggesting a more structured approach may result in greater clarity and transparency in formulary decision-making.