Overview:

This study protocol describes research to compare Twitter as an alternative quality measures data source to the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Adult Hospital Survey.

Background:

Patient Reported Measures (PRMs) have been the primary mechanism for integrating patient voices into quality measurement for clinical care, quality improvement, and accountability. But PRMs add burden to providers for implementation and data collection and to patients completing surveys. Social media may offer an alternative, but untested, data source for capturing patient experiences and outcomes that alleviates burden and broadens measurement capabilities.

Approach:

We will download 2019 CAHPS performance data for hospitals across the United States from the CMS data.medicare.gov website for the Overall Hospital Rating measure, “Patients who gave their hospital a rating of 9 or 10 on a scale from 0 (lowest) to 10 (highest)”. We will organize data by city, focusing on the average percent of patients in that city who rated their hospital as a 9 or 10.

We will then identify a population of patients posting on Twitter that mirrors the population of CAHPS respondents. We will select a sample of patient posts that describe a hospital visit and perform a qualitative analysis to identify lexical indicators of sentiment towards their experiences on a 1 (lowest) to 7 (highest) scale validated in other social media projects. We will review the resulting coding schema with patient/family co-researchers to ensure face-validity. Then, we will create algorithms to identify the sentiment expressed in relevant posts. We will parse the results by city, calculating the percent of patients posting about their hospital visits within the city who describe their hospital experience as very positive.

We will perform statistical analysis by city to assess the relationship between CAHPS measure performance and performance on the new social media measure.