Survey on Inter- and Intra-Observer Variations of the Ultrasound Assessment of Dog Pancreases

Abdominal ultrasonography (AUS) is considered an important diagnostic tool for assessing pancreatic disease in dogs. However, AUS interpretation is highly subjective and operator-dependent, with no prior studies evaluating observer variability in assessing the canine pancreas. This survey-based, observer agreement study aimed to assess inter- and intra-observer variability in the ultrasonographic evaluation of canine pancreases. Fifteen ultrasound images of dog pancreases were independently reviewed by 104 veterinary radiologists and radiology residents. Twenty-four veterinary radiologists repeated the review approximately 2.5 months later. Assessments included relative echogenicity, echotexture, margination, and clinical interpretation. Agreement was quantified using Krippendorff's alpha (Kα), prevalence-adjusted bias-adjusted kappa (PABAK), Cohen's kappa (κ), and intraclass correlation coefficient (ICC). Average inter-observer agreement was poor (Kα = 0.34), whereas intra-observer agreement was moderate (PABAK = 0.49). Global pancreatic echogenicity and echogenicity relative to mesenteric fat had the highest inter-observer agreement (Kα = 0.58 and 0.53, respectively) as did echotexture assessed by a semi-objective scale (Kα = 0.50). However, veterinary radiologists demonstrated poor inter-observer agreement on the clinical significance (Kα = 0.35) and prioritized cause (Kα = 0.14) of the pancreatic heterogeneity. Observer variables (age, training level, experience, primary practice, and accrediting organization) had minimal effect on inter-observer agreement, apart from experience, where early-career radiologists (<10 years' experience) demonstrated the highest inter-observer agreement among all groups, with moderate agreement for global echogenicity (Kα = 0.71). These findings highlight substantial variability in the ultrasonographic interpretation of the canine pancreas, particularly in clinical decision-making. Standardization of assessment criteria is necessary to improve consistency, clinical validity, and reduce diagnostic uncertainty.