Inter-rater agreement in the evaluation of abdominal radiographs for necrotizing enterocolitis
Sherif El-Kady1, Dara Petel1, Robert Baird1
1Department of Pediatric General and Thoracic Surgery, The Montreal Children's Hospital, McGill University, Montreal Quebec, Canada; McGill University, Montreal Quebec, Canada; Department of Radiology, The Montreal Children's Hospital, McGill University, Montreal Quebec, Canada.
Insights
Inter-rater agreement for necrotizing enterocolitis (NEC) diagnosis on abdominal radiographs varied among specialists. Radiologists showed excellent agreement, while agreement between surgeons and radiologists was only moderate, highlighting communication needs.
Area of Science:
- Neonatal imaging
- Pediatric radiology
- Surgical diagnostics
Background:
- Abdominal radiographs are crucial for necrotizing enterocolitis (NEC) surveillance in premature infants.
- Established findings exist, but inter-observer variability in interpretation is not well-studied.
Purpose of the Study:
- To evaluate the inter-rater agreement among radiologists and pediatric surgeons in interpreting abdominal radiographs for NEC.
- To assess agreement levels between attending physicians and trainees.
Main Methods:
- Thirty abdominal radiographs from premature infants with suspected NEC were reviewed.
- Attending radiologists, pediatric surgeons, and trainees independently assessed films for NEC findings.
- Fleiss's Kappa statistic was used to quantify inter-rater agreement.
Main Results:
- Radiologists demonstrated excellent agreement (Kappa=0.81).
- Agreement between surgeons and radiologists was moderate (Kappa=0.59).
- Agreement was poor for decubitus films, especially between surgeons and radiologists (Kappa=0.15).
Conclusions:
- While attending radiologists exhibit high agreement, inter-disciplinary agreement with surgeons is moderate.
- Trainee agreement with attendings was fair, suggesting a need for improved training and communication.
- Decubitus films may have limited utility in NEC evaluation due to poor agreement.
Purpose:
Abdominal radiographs are frequently employed in the surveillance of patients with necrotizing enterocolitis (NEC), with typical findings well described. Clinicians interpret and act upon these films at different intervals, however, and inter-rater agreement has not been evaluated to date.
Methods:
Thirty abdominal radiographs of premature infants were distributed to attending radiologists (4), pediatric surgeons (4), and trainees (4), who evaluated for findings of NEC from a list of eight potential choices (1=normal, 8=perforation). Fleiss's Kappa (FK) was used to evaluate concordance between multiple raters with 0-0.2=slight association and 0.8-1=almost perfect agreement.
Results:
Practicing surgeons had an FK of 0.77 overall (95% CI: 0.67-0.87), but demonstrated poor agreement when evaluating decubitus films (FK: 0.39, 95% CI:0.12-0.65). Radiologists had excellent inter-rater agreement (FK: 0.81, 95% CI: 0.74-0.88), but had only modest agreement with surgeons (FK: 0.59, 95% CI: 0.56-0.63) and poor agreement for decubitus films (FK: 0.15, 95% CI: 0.47-0.26). Surgical and radiology trainees had fair agreement with their respective attendings (0.60, 95% CI: 0.55-0.65 and 0.64, 95% CI: 0.60-0.69, respectively).
Conclusions:
While inter-rater agreement was good-excellent among attending staff, it was only moderate between radiologists and surgeons and between trainees and their attendings. This highlights the importance of inter-disciplinary and hierarchical communication to optimize clinical decision-making. Decubitus films may be of limited value in evaluating patients with NEC.
Related Concept Videos
01:25Assessment of the Abdomen I: Inspection and Auscultation
The abdominal examination is a cornerstone of clinical medicine, serving as a critical tool in diagnosing various gastrointestinal (GI) diseases. It involves a systematic approach that includes inspection and auscultation, each with distinct yet complementary roles in assessing the abdomen. This article will delve into these two primary methods healthcare professionals use to examine the abdomen.
Inspection of the Abdomen
The first step in any abdominal examination is inspection....
01:18Assessment of the Abdomen II: Percussion
Percussion
Percussion is an essential...
01:25Assessment of the Rectum and Anus
Rectal Inspection
Begin by inspecting the perianal and anal areas for color, texture, rashes,...
01:23Assessment of the Abdomen III: Palpation