Measuring Harm in Health Care: Optimizing Adverse Event Review

Area of Science:

• Healthcare quality improvement
• Patient safety research
• Clinical risk management

Background:

• Accurate assessment of healthcare-associated harm is crucial for clinical practice improvement and research.
• Reliability of clinician ratings for adverse events can be influenced by various factors, impacting data quality.

Purpose of the Study:

• To identify modifiable factors enhancing the reliability of health care-associated harm severity ratings.
• To optimize resource allocation for accurate harm assessment in clinical settings.

Main Methods:

• Utilized a generalizability theory framework to analyze rating reliability.
• Evaluated the impact of rater number, experience, and provider type on adverse event severity ratings.
• Included diverse clinician groups rating 8 types of adverse events.

Main Results:

• Pharmacists exhibited greater precision and consistency in harm severity ratings compared to physicians and nurses.
• Achieving a reliability of 0.83 could involve using 2 pharmacists instead of 3 physicians.
• Using two reviewers significantly improved reliability, with single raters miscategorizing ~5% of severe harm reviews.

Conclusions:

• Identified key factors influencing the reliability of clinician reviews for health care-associated harm.
• Generalizability analyses provided insights into optimizing rater selection for improved reliability and resource utilization.
• The approach offers potential applications for enhancing reliability in other healthcare assessment settings.