A Mixed Methods Study to Develop and Validate a National Patient Experience Survey (PES) to Support Service Delivery and Improvement
Speaker(s)
Withers K1, Palmer RI2, Woolley K3, Lewis J4, South K5, Desir R6
1CEDAR, Cardiff and Vale University Health Board, Cardiff, UK, 2CEDAR, Cardiff, CAY, UK, 3CEDAR, Cardiff, CRF, UK, 4NHS Wales Shared Services Partnership – Welsh Risk Pool, Cardiff, South Glamorgan, UK, 5elsh Value in Health Centre, Llantrisant, RCT, UK, 6Welsh Government, Cardiff, South Glamorgan, UK
Presentation Documents
OBJECTIVES: To develop a validated set of generic Patient Experience Questions to support service delivery and improvement
METHODS: This mixed methods study built on an existing set of PREMs. Interviews and focus groups were undertaken with 33 service users, advocacy group, and patient experience team representatives from across Wales. These discussions led to the development of a draft set of questions which were subsequently tested during iterative rounds of online surveys with almost 800 responses received in total. Data were analysed to test reliability and validity to confirm that:
- Questions were inclusive and easy to read / understand
- The tool avoids collecting an abundance of missing data.
- The tool is capable of measuring something, and not simply collecting random data (i.e. intra-rater reliability).
- The multiple questions are capable of collecting data that is consistent with each other, (i.e. internal consistency).
- The tool is capable of collecting data that is a good representation of the how they actually feel about their experience (i.e. criterion validity).
RESULTS: Eleven questions were initially developed in the qualitative phase with service users confirming face and content validity. All items except for question 8 “Did you need any help or support?”, showed good intra-rater reliability, with a Kappa or weighted Kappa coefficient of ≥0.75, and p values <0.005. There was strong criterion validity as measured against free text equivalents. Internal consistency produced an ordinal alpha, value of 0.92, with a 95% confidence interval of 0.78-0.98 for all but question 8. Question 8 was subsequently adapted and retested but again failed to meet adequate standards of validity and reliability and was therefore removed leaving 10 validated questions.
CONCLUSIONS: The People’s Experience Survey (PES) is a reliable and valid tool, suitable for use in any healthcare setting, and has been nationally approved to support service delivery and improvement.
Code
EPH29
Topic
Epidemiology & Public Health, Patient-Centered Research
Topic Subcategory
Instrument Development, Validation, & Translation, Patient Engagement, Public Health
Disease
Generics, No Additional Disease & Conditions/Specialized Treatment Areas