A SYSTEMATIC EVALUATION OF EMPIRICAL BAYES METHODS UNDER TIME CENSORED CONDITIONS
Author(s)
James F Murray, PhD, Director, Global Health Outcomes1, Dennis G Fryback, PhD, Professor21Eli Lilly and Company, Indianapolis, IN, USA; 2 University of Wisconsin-Madison, Madison, WI, USA
Presentation Documents
OBJECTIVES: Quality assurance must identify problems among a group of providers using sparse observations within some providers, posing problems of estimation. We compare Empirical Bayes (EB) methods for estimating means and confidence intervals to classical statistical estimates when time frames are censored. METHODS: Retrospective data on four measures with various units of analysis were selected: (1) quality of care for psychiatric emergencies by physicians, (2) cesarean sections rates in hospitals, (3) LOS for five DRGs in hospitals, and (4) “cost” for five DRGs by hospital. Data was available for overall time frames of one to two years. Individual unit of analysis results from the overall time frame were assumed as the “best” estimate of true performance. Data sets were subset into shorter time periods (i.e., months or quarters). EB priors were estimated by averaging results across units of analysis within each censored time frame. Priors were combined with the observed data to form posterior estimates for individuals; bootstrap methods were used to estimate Bayesian confidence intervals. Bayesian and Classical estimates of means and confidence intervals were compared within and across time periods by units of analysis on four measures of “goodness”: (1) Absolute difference between the “true” mean and the classical and EB estimates, (2) frequency of the EB mean being closest to the “true” mean, (3) average widths of the EB and classical intervals, and the actual probability coverage of the estimated intervals. RESULTS: EB methods were consistently closer in absolute difference and by frequency to the “true” mean. EB confidence intervals were consistently narrower giving less uncertainty about the location of the true mean while maintaining the expected probability coverage. CONCLUSION: Empirical Bayes methods can mitigate the adverse effect of sparse samples at the provider level compared to classical estimation for quality assurance when limited time constrains observations.
Conference/Value in Health Info
2009-05, ISPOR 2009, Orlando, FL, USA
Value in Health, Vol. 12, No. 3 (May 2009)
Code
MO2
Topic
Clinical Outcomes, Methodological & Statistical Research
Topic Subcategory
Clinical Outcomes Assessment, Modeling and simulation
Disease
Multiple Diseases