OBJECTIVES: Assess the use of validated cohort extraction algorithms (e.g., algorithms with known characteristics of sensitivity, specificity, etc.) from observational study literature.  We selected diseases the from 2012 CDC National Vital Statistics Report on Deaths amenable to retrospective cohort identifications. We targeted diagnosis, procedure and/or pharmacy-based algorithms used to identify patient cohorts; this abstract contains an initial focus on diabetes, pancreatic cancer, mild cognitive impairment (MCI) and migraine but the full analysis will have 16 targeted diseases. METHODS: We selected search terms to find articles that reported the actual criteria and algorithm used to identify a study cohort retrospectively. We searched in Medline on OVID platform using queries that combined diseases, study types and databases. The results were limited to human studies in United States in English published between 1990 and the present. One investigator assessed the retrieved studies against pre-determined search criteria for inclusion and exclusion. Two investigators independently assessed and classified the included studies into pre-defined categories of studies with unreported, non-validated algorithms and validated algorithms. We also extracted and collated the reported validation methods and operating characteristics. RESULTS: Literature search identified 441 studies for diabetes, 63 studies for pancreatic cancer, nine studies for MCI and 23 studies for migraine. Of these identified studies, 30.1%, 16.7%, 0% and 20% studies used validated algorithms for diabetes, pancreatic cancer, MCI and migraine respectively.   CONCLUSIONS: Algorithms extracted from the identified studies vary significantly in their nature (i.e., single codes or combinations of multiple codes), validation status and known operating characteristics. Use of non-validated algorithms can cause undesirable heterogeneity as well as the inability to validate and replicate findings. This can be a serious confounder and source of bias in cohort identification. The use of validated algorithms should be advocated and used over the development of new or ad hoc algorithms.