OBJECTIVES: Adverse symptom reporting is essential in clinical trials and drug labeling to assess and ensure patient safety. The standard approach to collecting adverse symptoms in cancer trials is clinician reporting using the Common Terminology Criteria for Adverse Events (CTCAE), which rates symptoms based upon descriptive clinical criteria. Despite the importance of this information, the reliability of these ratings has not been verified. At Memorial Sloan-Kettering Cancer Center, patients are routinely evaluated via CTCAE items by a clinician in an office suite, and again shortly thereafter by a second clinician in a chemotherapy suite, with no information passed between clinicians. METHODS: To measure the reliability of these evaluations, a retrospective analysis of medical charts was completed in a sample of 433 patients aged 26-91 (M = 62.39; 41.8% male) receiving chemotherapy, who were enrolled in an observational study conducted between March 2005 and August 2009. Cancer diagnoses included lung (N = 153), prostate (N = 127), and gynecologic (N = 153). RESULTS: For the first post-chemotherapy visit, intraclass correlation coefficients were moderate for fatigue (0.52), dyspnea (0.75), nausea (0.55), vomiting (0.50), diarrhea (0.63), constipation (0.48) and neuropathy (0.73). The average time between evaluations was 70.42 minutes (range 67.97-72.88).  These values were stable over up to six subsequent visits and did not differ based on age, gender, or elapsed time between evaluations. CONCLUSIONS: Given the short period of time and lack of interventions between reporting time points, the most likely cause of this lower than expected agreement between different clinicians is limited reliability of clinician reporting of this information. This finding has implications to clinical trials, as it brings into question the reliability or accuracy of symptom safety information. The investigators are currently evaluating patient-reported outcomes as an alternative and potentially more reliable method for collecting this information.