REDUCING COSTS AND IMPROVING OUTCOMES BY REDUCING MEDICAL ERRORS- A COMPARISON OF EXPERTS WITH PROBABLISTIC LABORATORY ERROR DETECTION IN A POPULATION OF PRE-DIABETICS
Author(s)
Jason N Doctor, PhD, Associate Professor1, Gregory Strylewicz, PhD, Software Engineer21University of Southern California, Los Angeles, CA, USA; 2 University of Washington, Seattle, WA, USA
Objective: Human evaluation of laboratory errors is a costly standard of practice. Automating error detection may reduce costs and improve patient outcomes. To compare an automated probabilistic approach (Bayesian network) to human expert error detection in a pre-diabetic population. Methods: Two test sets (A and B) each N=60 were generated from the results of the Diabetes Prevention Program (DPP). Glucose values were randomly drawn from a pre-diabetic distribution and expected HbA1c score was estimated by the DPP based formula: HbA1c=4.22 + 0.1604 x Glucose. In each test set, 37% of the HbA1c scores were mismatched to generate vial labeling errors. Eleven experts recruited from the American Academy of Clinical Chemists and a Bayesian network evaluated the results to detect mismatched vials. Six and five experts were assigned to test sets A and B respectively. Receiver-Operating Characteristics (ROC) curves were generated for each expert and for the Bayesian network and area under the curves (AUCs) were compared via null hypothesis testing. An AUC=1 and 0.5 represents perfect prediction and random guessing respectively. Results: The Bayesian network was predictive of glucose and HbA1c mismatches in both Test Set A (AUC = 0.86 (+/- 0.05)) and Test Set B (AUC = 0.93 (+/- 0.04)). Expert performance was on average worse in Test Sets A (AUC =0.74 (+/- 0.07)) and B (AUC=0.76 (+/- 0.07)). Individual analysis revealed that the network performed significantly better (z<1.96, p <0.05) than 7 of the 11 experts; in no case did the network perform worse than the experts. Conclusion: A Bayesian network that models probabilistic relationships among analyte values is often better than laboratory experts at identifying laboratory errors. This suggests that an automated program may help reduce costs and improve patient outcomes in the laboratory.
Conference/Value in Health Info
2008-05, ISPOR 2008, Toronto, Ontario, Canada
Value in Health, Vol. 11, No. 3 (May/June 2008)
Code
PDB80
Topic
Health Service Delivery & Process of Care
Topic Subcategory
Quality of Care Measurement
Disease
Diabetes/Endocrine/Metabolic Disorders