Related Experiment Video
Updated: Jan 14, 2026
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
The reliability of medical record review for estimating adverse event rates
Eric J Thomas1, Stuart R Lipsitz, David M Studdert
1Brigham and Women's Hospital and Harvard School of Public Health, Harvard University, Boston, Massachusetts 02115, USA.
Background:
The data used by the U.S. Institute of Medicine to estimate deaths from medical errors come from a study that relied on nurse and physician reviews of medical records to detect the errors.
Objective:
To measure the reliability of medical record review for detecting adverse events and negligent adverse events.
Design:
Medical record review.
Setting:
Hospitalizations in Utah and Colorado in 1992.
Measurements:
After three independent reviews of 500 medical records, the following were measured: reliability and the effect of varying criteria for reviewer confidence in and reviewer agreement about the presence of adverse events.
Results:
For agreements in judgments of adverse events among the three sets of reviews, the kappa statistics ranged from 0.40 to 0.41 (95% CIs ranged from 0.30 to 0.51) for adverse events and from 0.19 to 0.23 (CIs, 0.05 to 0.37) for negligent adverse events. Rates for adverse events and for negligent adverse events varied substantially depending on the degree of agreement and the level of confidence that was required among reviewers.
Conclusion:
Estimates of adverse event rates from medical record review, including those reported by the Institute of Medicine in its 2000 report on medical errors, are highly sensitive to the degree of consensus and confidence among reviewers.
Insights
Medical record review reliability for detecting adverse events is sensitive to reviewer agreement and confidence. Estimates of medical error rates, like those from the Institute of Medicine, vary significantly based on these factors.
Area of Science:
- Healthcare Quality and Safety
- Medical Informatics
- Patient Safety Research
Background:
- The U.S. Institute of Medicine (IOM) utilized nurse and physician reviews of medical records to estimate deaths from medical errors.
- This method relies on the accuracy and consistency of subjective reviews to identify errors within patient records.
Purpose of the Study:
- To quantify the reliability of medical record review processes for identifying adverse events.
- To assess the dependability of detecting specifically negligent adverse events through record review.
Main Methods:
- The study employed a medical record review design.
- Data were sourced from hospitalizations in Utah and Colorado during 1992.
- Five hundred medical records underwent three independent reviews to measure reliability and the impact of reviewer confidence and agreement criteria.
Main Results:
- Kappa statistics for agreement on adverse events ranged from 0.40 to 0.41, indicating moderate reliability.
- Agreement on negligent adverse events showed lower reliability, with kappa statistics from 0.19 to 0.23.
- Adverse event and negligent adverse event rates fluctuated considerably based on required reviewer consensus and confidence levels.
Conclusions:
- Adverse event rate estimations derived from medical record reviews are highly susceptible to the level of reviewer consensus and confidence.
- Findings suggest that previous estimates, including those by the IOM, may be influenced by reviewer agreement variability.
More Related Videos
06:55Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
10:46A Method of Trigonometric Modelling of Seasonal Variation Demonstrated with Multiple Sclerosis Relapse Data
Published on: December 9, 2015
Related Concept Videos
01:11Purpose of Health Records I
Here's a breakdown of how health records serve these purposes:
01:19Pharmacovigilance
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:20Introduction to Documentation and Reporting
Nursing documentation records essential information and details regarding a patient's care and treatment in written or electronic form. It is a critical aspect of nursing practice that involves documenting assessments, interventions, outcomes, and other relevant details about a patient's health status.
Documentation maps the patient's health journey by creating a comprehensive...
01:19Purpose of Health Records II
01:12Relative Risk