Reliability evaluation of the adapted national coordinating council medication error reporting and prevention (NCC...

Adapted National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index criteria were used in a study undertaken to evaluate commercial computerized provider order entry (CPOE) system impact on community hospital medication errors. This article describes: (1) adaptation of the Index, (2) classification criteria and processes used to assess the adapted Index, and (3) inter-rater reliability results.

A random sample of 130 (17%) of 2251 medication safety events (MSEs) were classified based on event type, that is, adverse drug event (ADE) or potential ADE (PADE); preventability, that is, 'yes' or 'no,' and outcome severity. Event outcome severity was categorized using adapted Index categories E-I (ADEs) and B-D (PADEs). Decision rules were used for rule-based classification, while an MSE Case Review Panel used judgment-based classification when decision rules did not apply. Inter-rater reliability for both classification approaches was assessed with kappa coefficients, percentage agreement, and confidence intervals (CI).

Level of agreement was substantial for both rule-based and judgment-based MSE classification for event type (6 = 0.70-0.90), preventability (6 = 0.67-0.82), and decision rule application (6 = 0.79). Rule-based agreement for ADE and PADE severity was almost perfect for discrete (6 = 0.83-0.84) and combined (6 = 0.87-0.90) Index categories. Judgment-based agreement was substantial for discrete (6 = 0.63-0.67) and combined (6 = 0.66-0.84) Index categories.

The adapted Index yielded substantial agreement for event type, preventability, and severity. Adaptation of the Index to support classification of non-preventable ADEs was an important improvement.