Inter-observer agreement in classifying anesthetic deaths in cats and dogs

Determining the cause of death in studies assessing mortality during small animal anesthesia poses challenges due to varying definitions of anesthetic death, limited information, and differences in evaluators' interpretations. This study aims to establish the interobserver agreement in classifying the cause of death in anesthetized animals. The observational, retrospective, multicenter study analyzed 432 deaths (83 cats and 349 dogs). Data were collected from a database of 55,022 anesthetized dogs and 14,962 anesthetized cats, created to investigate anesthetic-related mortality in these species. Three highly qualified veterinary anesthesiologists independently assessed whether the deaths were related to anesthesia, using their professional judgment. Data were collected from questionnaires that included the animal's signalment, reason for anesthesia, ASA status, drugs, anesthetic procedures, and comments from the submitting veterinarian. Light's Kappa and the intraclass correlation coefficient (ICC) measured interrater agreement among the three evaluators, while Cohen's Kappa assessed interrater reliability between pairs of observers (p < 0.05). Evaluators A, B, and C classified 296/432 (68.5%), 264/432 (61.1%), and 54/432 (12.5%) of the cases as anesthesia-related deaths, respectively. Agreement among the three evaluators was 128/432 (29.6%) [Light's Kappa: 0.17, p = 0.00026; ICC: 0.06, p-value = 0.0167]. The three evaluators agreed on classifying a death as anesthetic-related in 14.1% of cases (50 out of 354 cases where at least one evaluator classified the death as anesthetic-related). Similarly, for non-anesthetic-related deaths, the three evaluators reached an agreement in 20.4% of cases (78 out of 382 cases where at least one evaluator classified the death as non-anesthetic-related). Overall, agreement between two out of three evaluators was 304/432 (70.4%). Evaluators A and B had a 65.7% agreement [Cohen's Kappa: 0.25, p < 0.00001], A and C had a 46.6% agreement [Cohen's Kappa: 0.10, p < 0.00001], and B and C had a 50.9% agreement [Cohen's Kappa: 0.16, p < 0.00001]. In conclusion, the evaluators' agreement was weak, highlighting the need for a consensus on defining anesthetic mortality in dogs and cats.