OBJECTIVES: Observational studies are increasingly being used in comparative effectiveness research (CER) because they provide information on ‘real-world’ use of therapies.  Most decision makers, however, have little experience in evaluating observational research.  We assessed whether existing standards and scales were sufficient for identifying observational studies of high enough quality to inform decision making. METHODS: We identified standards and quality scales for observational studies published in the peer-reviewed literature or developed by regulatory agencies or other organizations. RESULTS: Many standards for observational studies exist, but most focus on pharmacovigilance or general good research practices.  Very few publications address the use of observational studies for CER, and only two – the U.S. AHRQ publication, “Registries for Evaluating Patient Outcomes:  A User’s Guide” and the GRACE Checklist – discuss this topic in detail.  The AHRQ registries guide provides a list of criteria to consider when evaluating the quality and appropriateness of an observational study, many of which are relevant to CER.  The GRACE Checklist is a validated scale to identify observational CER studies with sufficient quality to inform decision-making. CONCLUSIONS: Few standards and quality scales are available to assist decision makers in evaluating the quality of observational CER studies.  Once a baseline level of methodological rigor has been achieved, selecting specific studies to inform decision making is likely to be guided by nuanced factors related to the specific research question and decision at hand.  Rather than attempting to develop scales or standards that can independently be used for all types of research questions and decisions, a two-phase approach, in which a tool like the GRACE Checklist is used to identify studies of sufficient strength and then a framework is used to select the most appropriate studies for a specific research question, may be most useful to decision makers.