A validated, reliable method of scoring the severity of medication errors

Area of Science:

• Healthcare quality and safety
• Medication error analysis
• Psychometrics

Background:

• Medication errors pose significant risks to patient safety.
• Accurate severity scoring is essential for risk management and prevention.
• Existing methods may rely on patient outcomes, which are not always available.

Purpose of the Study:

• To develop and validate a method for scoring medication error severity independent of patient outcomes.
• To determine the reliability and generalizability of the proposed scoring method.
• To identify the optimal number of assessors required for reliable severity scores.

Main Methods:

• Developed a severity scoring scale (0-10) based on potential patient harm.
• 30 healthcare professionals from four UK hospitals scored 50 medication errors.
• Applied generalizability theory to analyze score variability from cases, judges, and scoring occasions.
• Included literature cases with known outcomes to assess score validity.

Main Results:

• Most score variance was due to differences between medication error cases.
• A generalizability coefficient of 0.8 or higher, indicating acceptable reliability, required at least four judges per case.
• Score reliability was not significantly affected by judge profession or the number of scoring occasions.
• The developed method proved reliable and valid for assessing medication error severity.

Conclusions:

• A reliable and valid method for scoring medication error severity without patient outcome data has been established.
• Utilizing a minimum of four judges per medication error is necessary to achieve reliable severity scores.
• The method's reliability is independent of the judges' professional backgrounds and the frequency of scoring occasions.