Inter-rater reliability between inexperienced and experienced raters using a rabbit sedation scale

To assess the reliability of a published rabbit sedation scale when used by inexperienced raters compared with experienced raters.

Experimental, randomised and blinded study.

A group of 16 rabbits, 3-4 months old and 1.9 ± 0.5 kg (mean ± standard deviation).

Rabbits were premedicated intramuscularly with midazolam 1 mg kg^-1 and butorphanol 0.5 mg kg^-1 (n = 8) or midazolam 1 mg kg^-1 and methadone 1 mg kg^-1 (n = 8). After a short explanation of the sedation scale, two experienced raters (one Diplomate of the European College of Veterinary Anaesthesia and Analgesia, and one Diplomate of the European College of Zoological Medicine) and four inexperienced raters (veterinary students) applied the sedation scale before sedation and at 10 and 20 minutes after sedation. Weighted kappa (κ_w) with linear weights and Gwet's agreement coefficient AC2 (Gwet's AC2) were used to determine inter-observer agreement between the two Diplomates, and Gwet's AC2 and Krippendorff's α (α_k) coefficients were used to evaluate inter-observer agreement between the four students for each of the eight items. Agreement was considered when κ_w = 0.81-1.000. Lin's concordance correlation coefficient was used to evaluate the concordance. A paired t-test and Gwet's AC2 coefficient were used to compare the groups.

There was agreement within experienced [κ_w = 0.981 (0.970-0.992), p = 0.001; Gwet's AC2 = 0.973 (0.943-1.000), p = 0.001] and inexperienced raters [α_k = 0.658 (0.618-0.699), p = 0.001; Gwet's AC2 = 0.741 (0.705-0.778), p = 0.001]. There were statistically significant differences depending on the experience of the rater (p = 0.001). The sedation protocol did not affect the agreement between raters (p = 0.084).

The rabbit sedation scale was reliable when used by experienced or inexperienced raters.