Asking residents about adverse events in a computer dialogue: how accurate are they?

Although retrospective identification of adverse events is time-consuming, whether they are present and/or expected is often readily apparent to providers during the provision of care.

A computer program to flag admissions with possible adverse events was developed. Readmissions to the hospital within 31 days and admissions including more than one visit to the operating room (OR) were flagged. For surgical site infections, all admissions--including a visit to the OR--were flagged, but only a sample was evaluated in the reliability assessment. Residents in an urban, tertiary care hospital were questioned when inputting computerized discharge orders regarding adverse events among 391 cases sampled from 6,813 admissions for a two-month period.

For the 228 readmissions (3.3% of all admissions) identified by the computer program, resident responses had a sensitivity of 57% and a specificity of 73% in detecting an unexpected readmission (nurse responses, 96% and 91%). For the 79 patients with a return to the OR, the residents' responses had a sensitivity of 86% and a specificity of 84% for detecting an unexpected return (versus 75% and 98% for the nurses' responses). For the 209 patients with an OR visit, the sensitivity and specificity for a surgical site infection were 85% and 98% for the residents and 54% and 99% for the nurses.

Information systems can be used to screen for adverse events and to ask providers whether adverse events are unexpected, although the reliability of this approach is likely to vary by event type.