THE IMPORTANCE OF METHODOLOGY IN REPORTING PERFORMANCE INDICATORS
Author(s)
Chen M1, Levinton C2
1, 2Health Quality Ontario, Toronto, ON, Canada
Presentation Documents
OBJECTIVES: The aim of this research is to use two public reporting methods to compare and contrast institutional performance by analyzing, as an example, a patient satisfaction indicator, namely, percentage of patients who would definitely recommend the emergency department (ED) to their friends and family. METHODS: There are a multitude of ways to evaluate hospital results on patient satisfaction in ED. A standard approach sets a target to gauge hospital performance. Values above the target/benchmark meet or exceed expectations, while those institutions below the target may consider initiatives and programs to improve their patient satisfaction score. An alternative method is the use of funnel plots that assess the relationship between the outcome measure (patient satisfaction score) and volume of cases across disparate facilities when evaluating institutional performance. Allowances are made for hospitals falling within a band, called control limits, that adjusts for precision. RESULTS: From April 2010 to March 2011, 99 Ontario hospitals reported their scores of a NRC-Picker patient satisfaction survey question. A funnel plot was constructed using a benchmark (70.6%). In all, 45(45%) institutions performed within expected variation, that is within or above the 99.8% control limit, and 54(55%) fell below the lower 99.8% control limit. When we simply use the 70.6% target without any control limits, 19 facilities exceeded the threshold, while 80 fell below it. Divergence in reporting occurred where 26 institutions met the 99.8% control limits set by the funnel plot, but fell below the benchmark set when using a fixed target of 70.6%. CONCLUSIONS: The two approaches lead to different conclusions on hospital performance for 26 (26%) institutions. The first method compared hospital scores against a target or preset benchmark while the second approach used funnel plots to interpret institutional performance with greater precision by taking into account variation via sample size.
Conference/Value in Health Info
2014-05, ISPOR 2014, Palais des Congres de Montreal
Value in Health, Vol. 17, No. 3 (May 2014)
Code
PHS89
Topic
Patient-Centered Research
Topic Subcategory
Stated Preference & Patient Satisfaction
Disease
Multiple Diseases, Reproductive and Sexual Health