AI-Estimated Depression Scores as Real-World Evidence of Treatment-Resistant Depression

Pedro Alves; Carl D. Marci, MD; Costas Boussios, PhD, OM1, Inc., Boston, MA, USA

Major depressive disorder (MDD) is one of the most prevalent and disabling conditions worldwide with widespread economic impact, yet measuring its severity in routine clinical practice remains surprisingly difficult. The Patient Health Questionnaire 9-item (PHQ-9) is the standard tool for quantifying depressive symptoms, but clinicians often omit it from the medical record, leaving large gaps in real-world data (RWD) and no objective markers of disease severity. For health economists, outcomes researchers, and healthcare decision makers who rely on RWD to track treatment effectiveness, this gap is more than an inconvenience—it is a fundamental limitation to advancing therapies and knowledge.

Our team of researchers at OM1, Inc. set out to fill this gap. In a study presented at ISPOR Europe 2025 in Glasgow, the team tested whether an artificial intelligence model called the ePHQ-9—which infers PHQ-9 depression severity scores directly from clinical notes in electronic medical records (EMRs)—could serve as a reliable proxy when a conventional PHQ-9 score is absent. We aimed to determine the usefulness of ePHQ-9 for identifying one of the most clinically meaningful presentations of severe MDD, namely treatment-resistant depression (TRD).

The ePHQ-9 estimates what a patient’s depressive symptom score would be based on the language clinicians use in their
notes to describe the patient’s recent history, symptoms, and course of illness.

The PHQ-9 Problem in Real-World Data

The PHQ-9 is a 9-item questionnaire, typically completed by the patient, that scores depressive symptoms on a 0–27 scale across 5 severity categories:

• None-Minimal (PHQ-9 = 0-4)

• Mild (PHQ-9 = 5-9)

• Moderate (PHQ-9 = 10-14)

• Moderate-Severe (PHQ-9 = 15-19)

• Severe (PHQ-9 = 20-27)

Clinicians and researchers alike regard PHQ-9 as a gold standard for evaluating depression severity and treatment response. The problem is that it only appears in the medical record when someone remembers—or chooses—to administer it. In large, population-level real-world datasets, this is far from guaranteed, with many patients who have only a few observed scores over time and others with no severity scores at all.

The ePHQ-9 was developed to help fill the gaps in real-world measurement. Using a previously published machine learning model trained on clinical notes from a large condition specific dataset, it estimates what a patient’s PHQ-9 score would be based on the language clinicians use to describe their recent history, symptoms, and course of illness. A published validation study demonstrated strong analytic performance of the model in estimating PHQ-9 scores from both structured (ie, templated) and unstructured (ie, free text) note elements. The current study takes the next step: Does the ePHQ-9 reflect clinically meaningful variation in disease burden?

Treatment-Resistant Depression: A Meaningful Benchmark

TRD—broadly defined as MDD that fails to respond adequately to at least 2 antidepressant treatments of adequate dose and duration—is a well-recognized clinical entity affecting a substantial proportion of patients with MDD.² Critically, TRD has no ICD diagnostic code despite the fact that they tend to suffer with more severe and more debilitating symptoms compared with depressed patients without TRD. Identifying patients with TRD from real-world datasets requires natural language processing (NLP) of clinical notes to find physician attestations of the condition—exactly the kind of rich clinical signal that the ePHQ-9 was built to harness. This makes TRD an ideal real-world benchmark for testing a severity measure derived from clinician narratives.

Study Design   

The study drew on a large, curated US real-world dataset of patients with MDD (OM1, Inc., Boston, MA). The dataset combines linked claims and EMR data with both structured and unstructured clinical notes. The final cohort comprised 76,873 patients, of whom 28,641 had an observed PHQ-9 score and 61,602 had an ePHQ-9 score recorded within 30 days of their index date.

Physician-attested TRD was identified via NLP of clinical notes (for example, “Continue Abilify to 7.5 mg PO QHS for treatment-resistant depression symptoms and irritability/agitation”), with human coders reviewing a subset as validation. We found that 1699 patients were labeled as TRD, including 1082 with an observed PHQ-9 and 1232 with an ePHQ-9 score. To evaluate the relationship between depression severity and TRD, the team examined the proportion of TRD patients within each of the 5 PHQ-9 severity categories.

A Clear Signal: Stronger Relationship in the AI Model

Both the observed PHQ-9 and the ePHQ-9 revealed a modest, monotonic pattern: As depression severity increased, so did the proportion of patients with physician-attested TRD (Figures 1 and 2). This is exactly what one would expect if both measures are genuinely capturing disease burden.

The difference lies in the strength of that association. For the observed PHQ-9, TRD prevalence rose from 9% in the None-Minimal category to 36.6% in the Severe category, yielding a Pearson correlation of r = 0.24. The ePHQ-9 told a more vivid story: TRD prevalence ranged from 4.2% in the None-Minimal group to 52.7% in the Severe category, with a stronger correlation of r = 0.40. Put differently, more than half of all patients classified as severely depressed by the ePHQ-9 also carried a physician attestation of TRD—a convergence that supports the clinical relevance of the AI-derived measure.

Figure 1. Physician-Attested TRD Proportion by PHQ-9 Severity

Figure 2. Physician-Attested TRD Proportion by ePHQ-9 Severity

Why Does the AI Model Outperform?

The finding that the ePHQ-9 correlates more strongly with TRD than the observed PHQ-9 may seem counterintuitive given the ubiquitous use of the PHQ-9. The explanation likely lies in the source of the data. The PHQ-9 is a patient-reported outcome completed at a single point in time, often in a waiting room or asynchronously online prior to a clinical encounter. The score reflects their subjective assessment of symptoms over the prior 2 weeks. As such, it reflects only the patient perspective and may not capture the full clinical picture.

The ePHQ-9, by contrast, is derived from clinician narratives: the language physicians use when documenting a patient’s history. This includes clinical judgment combined with patient reports of symptom burden, treatment course, and response over time. Because TRD is also identified through physician attestation in clinical notes, the 2 measures share a common clinical “vocabulary,” producing a strong alignment. Both the ePHQ-9 and TRD attestation, in effect, speak the language of the treating physician, which may account for the results of this study.

Any tools that extend the reach and fidelity of real-world evidence are not merely methodological conveniences—they are essential to informed decision making.

Implications for HEOR and RWE

These findings carry practical implications for health economics and outcomes research. TRD is expensive, burdensome, and increasingly the target of novel therapeutic interventions. Yet studying it in real-world settings has been hampered by the absence of a diagnostic code and the limited availability of validated severity measures interrogated into EMR data.

The ePHQ-9 offers a potential solution on both counts. With its ability to generate depression severity estimates across a far larger patient population than the observed PHQ-9 (61,602 versus 28,641 patients in this study), and its stronger alignment with clinically attested treatment resistance, it may enable more robust and comprehensive RWD research into MDD and its most refractory forms.

While the RWD in this study were limited by documentation quality and consistency and the potential for selection bias, the findings demonstrate the potential value of AI-enabled innovations like the ePHQ-9. For healthcare decision makers evaluating comparative effectiveness, health technology assessments, and coverage policies, any tools that extend the reach and fidelity of real-world evidence are not merely methodological conveniences—they are essential to informed decision making.

Lessons Learned

This study represents a meaningful step in the validation of AI-derived clinical measures for use in RWD research. While more research is needed involving additional datasets from additional health networks, the ePHQ-9’s stronger correlation with physician-attested TRD than the conventional patient-reported PHQ-9 underscores the potential value of mining clinician narratives for outcomes research.

As EMR data grow richer and NLP methods continue to mature, studies like this help support a new generation of AI-enabled measures that can fill the gaps where traditional instruments fall short.