Related Experiment Video
Updated: Dec 13, 2025
Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
Published on: June 12, 2019
Inter-rater Reliability of Sacral Ratio Measurements in Patients with Anorectal Malformations
Gregory Metzger1, Jennifer N Cooper1, Rashmi S Kabre2
1Department of Pediatric Surgery and the Research Institute, Nationwide Children's Hospital, Columbus, Ohio.
Insights
Radiologists showed excellent agreement in calculating the sacral ratio (SR) from anteroposterior (AP) X-rays for children with anorectal malformations (ARM). However, agreement was poor for lateral X-rays, indicating a need for standardized measurement protocols.
Area of Science:
- Pediatric Radiology
- Congenital Anomalies
- Skeletal Development Assessment
Background:
- Anorectal malformations (ARMs) present a spectrum of congenital anomalies impacting fecal continence.
- The sacral ratio (SR) is a proposed metric for predicting fecal continence in pediatric ARM patients.
- Quantifying inter-rater reliability (IRR) of SR measurements is crucial for clinical application.
Purpose of the Study:
- To assess the inter-rater reliability (IRR) of sacral ratio (SR) calculations among radiologists.
- To evaluate SR measurement consistency across different institutions for pediatric anorectal malformation (ARM) assessment.
Main Methods:
- Review of X-rays (anteroposterior and lateral) from 263 pediatric patients with ARM.
- Independent SR calculations by a radiologist from each of six institutions, followed by central review.
- Statistical analysis using Pearson correlation and intraclass correlation coefficients (ICCs) from mixed-effects models.
Main Results:
- Excellent inter-rater reliability (ICC, 81.5%) was observed for AP sacral ratio (SR) calculations.
- Poor inter-rater reliability (ICC, 44.0%) was found for lateral sacral ratio (SR) measurements.
- Reliability remained consistent across varying ARM severity, spinal anomalies, and patient age.
Conclusions:
- Sacral ratio (SR) calculations demonstrate high reliability in the AP plane for anorectal malformation (ARM) assessment.
- Lateral SR measurements exhibit poor reliability, suggesting a need for improved standardization.
- Standardized protocols for lateral SR are essential for its reliable use in counseling families of children with ARM.
Background:
Anorectal malformations (ARMs) are a spectrum of congenital anomalies with varying prognosis for fecal continence. The sacral ratio (SR) is a measure of sacral development that has been proposed as a method to predict future fecal continence in children with ARM. The aim of this study was to quantify the inter-rater reliability (IRR) of SR calculations by radiologists at different institutions.
Materials And Methods:
x-Rays in the anteroposterior (AP) and lateral planes were reviewed by a pediatric radiologist at each of six different institutions. Subsequently, images were reviewed by a single, central radiologist. The IRR was assessed by calculating Pearson correlation coefficients and intraclass correlation coefficients from linear mixed models with patient and rater-level random intercepts.
Results:
Imaging from 263 patients was included in the study. The mean inter-rater absolute difference in the AP SR was 0.05 (interquartile range, 0.02-0.10), and in the lateral SR was 0.16 (interquartile range, 0.06-0.25). Overall, the IRR was excellent for AP SRs (intraclass correlation coefficient [ICC], 81.5%; 95% confidence interval, 75.1%-86.0%) and poor for lateral SRs (ICC, 44.0%; 95% CI, 29.5%-59.2%). For both AP and lateral SRs, ICCs were similar when examined by the type of radiograph used for calculation, severity of the ARM, presence of sacral or spinal anomalies, and age at imaging.
Conclusions:
Across radiologists, the reliability of SR calculations was excellent for the AP plane but poor for the lateral plane. These results suggest that better standardization of lateral SR measurements is needed if they are going to be used to counsel families of children with ARM.
