USABILITY EVALUATION OF CLINICAL AND OPERATIONAL POPULATION MANAGEMENT TOOLS IN VISN 21
Author(s)
Bena JM1, Taylor J1, Carmichael J1, Meier J2, Robinson A3
1VA Sierra Pacific Network (VISN 21), Reno, NV, USA, 2VA Sierra Pacific Network (VISN 21), Martinez, CA, USA, 3VA Patient Education Resource Center, Palo Alto, CA, USA
OBJECTIVES: Veteran Integrated Service Network (VISN) 21 Pharmacy Benefits Management (PBM) pharmacists utilize electronic health record data to create and maintain over 300 real-time clinical and operational population management dashboards and reports. These tools allow for a comprehensive and interactive view and analysis of pharmacotherapeutic and performance data. Clinicians and managers use this data to more efficiently monitor patient populations and staff for quality, safety, and value. While anecdotal feedback has been solicited and use of these tools has been widely adopted in VISN 21, utility has never been formally assessed. This project will evaluate the current usability of these tools. METHODS: A survey based on the validated System Usability Scale (SUS) to objectively measure usability was developed. The instrument gathers demographic and use information from current users, opinions on data accuracy, ease, and applicability of the current tools and was sent to staff that had accessed a dashboard in the prior year. Following this survey, an in-depth heuristic evaluation will be done by external dashboard developers. Changes will be made to dashboards based on feedback and the SUS survey instrument re-sent to assess any change in usability. RESULTS: For the first SUS survey we received 207 responses out of 1653 surveyed (12.5 % response rate), with 73 pharmacists (35%), 72 nurses (35%), and 26 physicians (13%), 36 others (17%) responding. SUS average scores for the 5 most used dashboards were 61.1±14.84 (N=151). On a scale of 1-5, with 1 being strongly disagree and 5 being strongly agree, user confidence that data matches the medical record was 3.43±1.00, accurate and correct 2.62±1.15, and most current 3.71±1.07. CONCLUSIONS: Respondents considered dashboard usability marginally acceptable by SUS grading scale. Users agreed that the dashboard and report data matches the medical record and is up-to-date. Next steps are to complete the heuristic evaluation, make changes, and re-survey.
Conference/Value in Health Info
2017-05, ISPOR 2017, Boston, MA, USA
Value in Health, Vol. 20, No. 5 (May 2017)
Code
PRM51
Topic
Real World Data & Information Systems
Topic Subcategory
Reproducibility & Replicability
Disease
Multiple Diseases