Variability in Grading Diabetic Retinopathy Using Retinal Photography and Its Comparison with an Automated Deep...

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence in Healthcare

Background:

• Diabetic retinopathy (DR) screening via color retinal photographs is efficient.
• DR severity grading in clinical practice involves graders with diverse expertise.
• Variability in human grading necessitates evaluation of automated systems.

Purpose of the Study:

• To assess agreement in DR severity grading between human graders of varying expertise and an automated deep learning screening software (ADLS).
• To compare the diagnostic performance of ADLS against human grading for referable DR.

Main Methods:

• Two hundred fundus photographs were graded using the International Clinical DR Disease Severity Scale by retinal specialists, residents, physicians, students, and ADLS.
• Grading outcomes were categorized into no referral, non-urgent referral, and urgent referral.
• Inter-observer and intra-group variability were analyzed using Gwet's agreement coefficient; ADLS performance was assessed via sensitivity and specificity.

Main Results:

• Inter-observer agreement ranged from fair to very good; intra-group agreement ranged from moderate to good.
• ADLS demonstrated high area under the curve values (0.879 for non-referable DR, 0.714 for non-urgent referable DR, 0.836 for urgent referable DR).
• Sensitivity and specificity values for ADLS varied across different referral categories.

Conclusions:

• Human grading of DR severity exhibits significant inter-observer and intra-group variability.
• ADLS proves to be a reliable and sensitive tool for large-scale screening of referable DR.
• The automated system aids in identifying patients requiring urgent ophthalmological referral for diabetic retinopathy.