OBJECTIVES : This study examined change over time in depression with standard sum vs. item response theory (IRT) scoring among people initiating treatment for depression. We compared measures of clinically meaningful and statistically significant change.

METHODS : Data on Patient Health Questionnaire 9 (PHQ9) item responses was extracted from the electronic health records of 5,405 people receiving depression treatment in three health systems at the start of treatment and 30 to 180 days later. We used three methods to classify change for sum scores: the reliable change index (RCI), the 5-point change and 50% change from baseline. For IRT scores, we used the z-test to determine statistically significant change.

RESULTS : Follow-ups occurred a median of 9 weeks (interquartile range: 6-16 weeks) after treatment started. The percent agreement between the RCI (92.2%), 5-point change (83.6%) or 50% change (83.6%) with the z-test for IRT scoring was acceptable. While the kappa between the RCI and z-test was adequate (0.81), the kappas between the z-test and the 5-point change (0.64) and the 50% change (0.62) were low. Change using the 5-point change or 50% change but not IRT scoring (16.2%, 10.2%) was more common than no change using 5-point or 50% change but change using IRT scoring (0.3%, 6.3%). The mean absolute symptom change in the groups where IRT scoring and sum scoring disagreed ranged from 3.5 to 7.6.

CONCLUSIONS : IRT and sum scoring disagree at the individual level both in showing no change had occurred and in showing change had occurred (the treatment was working). Clinicians may have the most difficulty interpreting scores in the range of small or moderate symptom change and this is where IRT and sum scores gave different information.