OBJECTIVE: Numerous health outcomes research studies utilize health plan administrative claims data to identify various disease states. However, limited studies have evaluated the diagnostic validity of these data sources for this purpose. This study was conducted to evaluate the concordance between claims-based identification rules and medical records (MR) for coronary heart disease (CHD), diabetes, and obesity. METHODS: A random sample of patients on statin therapy from a West Coast health plan with benefit eligibility from January 1, 2002 to July 31, 2004 was identified via administrative claims data. All diseases in these patients were identified using medical claims. MR were utilized as the “gold-standard” and were abstracted by trained MR reviewers using a standardized abstraction form over the eligibility date range. Sensitivity/specificity and kappa coefficients were calculated to examine the validity of claims and the agreement between the two data sources, respectively. RESULTS: A total of 531 patient MR were abstracted (15% CHD, 21% diabetes, and 23% obesity via MR). Diabetes had the highest sensitivity at 96%, followed by CHD and obesity at 78% and 18%, respectively. However, obesity demonstrated the highest specificity (97%), with diabetes at 93%, and CHD at 87%. Agreement between claims data and MR was good for diabetes (kappa=0.82, 95% CI=0.77-0.88), moderate for CHD (kappa=0.53, 95% CI=0.44-0.62), and poor for obesity (kappa=0.20, 95% CI=0.11-0.29). CONCLUSION: In our study, administrative claims data were found to have acceptable sensitivity, specificity and agreement for diabetes and CHD. These data suggest that claims data are a viable data source for health outcomes research for those disease states. For less/poorly coded conditions such as obesity, medical records (combined with administrative claims data) may be a more valid data source.