The Institute of Medicine Report on Patient Safety requests a 50% reduction of adverse events. This however, poses a challenge from a measurement perspective. Incidence estimates of adverse events have traditionally been based on medical case review, which is exhaustive and costly. Furthermore, the reliability of this method has been challenged. Recent studies have applied information technology to screen for adverse drug events (ADEs) in order to replace chart review. OBJECTIVE: To explore the extent to which automated health care data screens can be used to measure the incidence of in-hospital ADEs. METHODS: Systematic literature review. Keyword search of MEDLINE (1980 - 2001) electronic database and hand search. Included studies had to describe specific screens that were applied to clinical or administrative databases to detect in-hospital ADEs. They also had to formally evaluate the validity of the screens with regards to their ability to correctly identify ADEs. Validity was expressed as positive predictive value (PPV). RESULTS: Ten studies published between 1992 and 2000 met the inclusion criteria. Three approaches used to measure ADE incidence were identified. Two studies screened for generic adverse outcomes (e.g., inpatient deaths), the average PPVs were 1% and 17.4%. Five studies exclusively screened for surrogate outcomes (antidotes commonly used to treat ADEs, or critical lab values, such as elevated creatinine or drug levels) to predict the occurrence of an ADE, with PPVs of 9, 12, 13, 18 and 37%. Three studies tested screens that combined medications and intermediate outcomes (PPVs 12.4, 45 and 53%). CONCLUSIONS: Automated health care data screens show promise as ADE incidence measure. Their current validity, however, does not appear to be sufficient for cross-sectional comparisons or the evaluation of quality improvement initiatives. Increasing sophistication of the screens by including multiple variables that link process components (e.g. medication) along with adverse outcomes or surrogates (e.g. lab values, antidotes) appear to increase screen validity.