Measuring Harm in Health Care: Optimizing Adverse Event Review
Related Experiment Video
Updated: Mar 11, 2026
Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Kathleen E Walsh1, Polina Harik, Kathleen M Mazor
1*James M. Anderson Center †Department of Pediatrics, Cincinnati Children's Hospital, Cincinnati, OH ‡National Board of Medical Examiners, Philadelphia, PA §Meyers Primary Care Institute, Worcester, MA ∥Agency for Healthcare Quality and Research, Rockville, MD ¶Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester #Kindred Healthcare, Wellesley, MA.
Objective:
The objective of this study was to identify modifiable factors that improve the reliability of ratings of severity of health care-associated harm in clinical practice improvement and research.
Methods:
A diverse group of clinicians rated 8 types of adverse events: blood product, device or medical/surgical supply, fall, health care-associated infection, medication, perinatal, pressure ulcer, surgery. We used a generalizability theory framework to estimate the impact of number of raters, rater experience, and rater provider type on reliability.
Results:
Pharmacists were slightly more precise and consistent in their ratings than either physicians or nurses. For example, to achieve high reliability of 0.83, 3 physicians could be replaced by 2 pharmacists without loss in precision of measurement. If only 1 rater was available for rating, ∼5% of the reviews for severe harm would have been incorrectly categorized. Reliability was greatly improved with 2 reviewers.
Conclusions:
We identified factors that influence the reliability of clinician reviews of health care-associated harm. Our novel use of generalizability analyses improved our understanding of how differences affect reliability. This approach was useful in optimizing resource utilization when selecting raters to assess harm and may have similar applications in other settings in health care.
Related Concept Videos
01:19Pharmacovigilance
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
01:11Purpose of Health Records I
Here's a breakdown of how health records serve these purposes:
01:21Types of Reports II: Incident or Occurrence Report
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
01:15Methods of Documentation VI: Case Management Model
For example, a patient with a chronic...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:30Ethical Dilemmas II