Assessing intergrader variability in incomplete outer retinal atrophy (iRORA) grading

Area of Science:

• Ophthalmology
• Medical Imaging
• Biomarker Research

Background:

• Incomplete outer retinal atrophy (iRORA) is a key biomarker for early age-related macular degeneration (AMD).
• Accurate detection of iRORA is vital for monitoring disease progression and evaluating treatments.
• Current grading criteria present challenges for consistent intergrader assessment.

Purpose of the Study:

• To evaluate the intergrader variability in detecting iRORA using optical coherence tomography (OCT) scans.
• To assess the reliability of iRORA as a biomarker and potential clinical trial endpoint.

Main Methods:

• A cross-sectional study involving five international retinal experts grading 60 OCT B-scans for iRORA.
• Utilized the Classification of Atrophy Meeting (CAM) criteria and a standardized online annotation platform.
• Assessed intergrader agreement using Cohen's Kappa and analyzed classification accuracy, sensitivity, and specificity.

Main Results:

• Mean pairwise Cohen's Kappa was 0.425, indicating mild agreement among graders.
• Agreement with the majority vote ranged from κ=0.48 to 0.67, with classification accuracy between 0.73 and 0.83.
• High specificity (0.85-0.96) was observed, but sensitivity varied (0.59-0.83).

Conclusions:

• Intergrader reproducibility for iRORA detection remains limited despite standardized CAM definitions.
• Challenges exist in consistently identifying early atrophic changes in AMD.
• Refining grading guidelines and incorporating automated detection systems are recommended to enhance reliability.