OBJECTIVES: The GRACE checklist is a tool for evaluating the quality of comparative effectiveness research (CER) studies.  The checklist consists of 11 questions on data and methods and was developed through literature review, expert consultation, and testing by 113 raters across five continents.  The purpose of the present research was to determine which checklist questions are most predictive in identifying quality CER. METHODS: Twenty-two volunteers recruited from academia, industry, and government applied the GRACE checklist to 28 CER articles, for a total of 56 assessments. We used Classification and Regression Tree (CART) methodology, a binary, recursive, partitioning methodology, to identify the checklist questions that were most predictive of quality CER articles.  Quality was defined as a composite outcome of three indictors: journal impact factor, article citations per year, and expert assessment of whether the research was designed and executed well enough to be reliable.  An article was considered to meet all three quality criteria if journal impact factor was higher (>2.5), there were frequent article citations per year (>2), and the expert assessment of overall quality was classified as “sufficient”. RESULTS: The CART analysis revealed high sensitivity (71.43%, i.e., ability to detect a "quality" CER article, based upon the composite score) and high specificity (80.95%, i.e., ability to identify a CER article not meeting sufficient quality standards).  The use of a composite outcome in the CART analysis yielded higher sensitivity and specificity than any of the outcomes individually.  Alteration of the default tree settings by varying the "penalty" for misclassifying sufficient vs. non-sufficient quality articles had minimal impact on the sensitivity and specificity.  The use of a sensitivity analysis was the strongest predictor of quality.    CONCLUSIONS: The high specificity of the tree indicates the checklist, particularly the sensitivity analysis question, can be used to identify articles that do not meet a sufficient quality standard.