OBJECTIVES: To develop a validated set of generic Patient Experience Questions to support service delivery and improvement

METHODS: This mixed methods study built on an existing set of PREMs. Interviews and focus groups were undertaken with 33 service users, advocacy group, and patient experience team representatives from across Wales. These discussions led to the development of a draft set of questions which were subsequently tested during iterative rounds of online surveys with almost 800 responses received in total. Data were analysed to test reliability and validity to confirm that:

• Questions were inclusive and easy to read / understand
• The tool avoids collecting an abundance of missing data.
• The tool is capable of measuring something, and not simply collecting random data (i.e. intra-rater reliability).
• The multiple questions are capable of collecting data that is consistent with each other, (i.e. internal consistency).
• The tool is capable of collecting data that is a good representation of the how they actually feel about their experience (i.e. criterion validity).

RESULTS: Eleven questions were initially developed in the qualitative phase with service users confirming face and content validity. All items except for question 8 “Did you need any help or support?”, showed good intra-rater reliability, with a Kappa or weighted Kappa coefficient of ≥0.75, and p values <0.005. There was strong criterion validity as measured against free text equivalents. Internal consistency produced an ordinal alpha, value of 0.92, with a 95% confidence interval of 0.78-0.98 for all but question 8. Question 8 was subsequently adapted and retested but again failed to meet adequate standards of validity and reliability and was therefore removed leaving 10 validated questions.

CONCLUSIONS: The People’s Experience Survey (PES) is a reliable and valid tool, suitable for use in any healthcare setting, and has been nationally approved to support service delivery and improvement.