Related Experiment Video
Updated: Jun 3, 2025
Artificial Intelligence Approaches to Assessing Primary Cilia
Published on: May 1, 2021
The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
Hiroki Maehara1,2, Yuta Ueno3,4, Takefumi Yamaguchi5,2
1Department of Ophthalmology, Fukushima Medical University School of Medicine, Fukushima, Japan.
Abstract:
We developed an AI system capable of automatically classifying anterior eye images as either normal or indicative of corneal diseases. This study aims to investigate the influence of AI's misleading guidance on ophthalmologists' responses. This cross-sectional study included 30 cases each of infectious and immunological keratitis. Responses regarding the presence of infection were collected from 7 corneal specialists and 16 non-corneal-specialist ophthalmologists, first based on the images alone and then after presenting the AI's classification results. The AI's diagnoses were deliberately altered to present a correct classification in 70% of the cases and incorrect in 30%. The overall accuracy of the ophthalmologists did not significantly change after AI assistance was introduced [75.2 ± 8.1%, 75.9 ± 7.2%, respectively (P = 0.59)]. In cases where the AI presented incorrect diagnoses, the accuracy of corneal specialists before and after AI assistance was showing no significant change [60.3 ± 35.2% and 53.2 ± 30.9%, respectively (P = 0.11)]. In contrast, the accuracy for non-corneal specialists dropped significantly from 54.5 ± 27.8% to 31.6 ± 29.3% (P < 0.001), especially in cases where the AI presented incorrect options. Less experienced ophthalmologists were misled due to incorrect AI guidance, but corneal specialists were not. Even with the introduction of AI diagnostic support systems, the importance of ophthalmologist's experience remains crucial.
Insights
AI
Area of Science:
- Ophthalmology
- Artificial Intelligence
- Medical Diagnostics
Background:
- Anterior eye image classification using AI is emerging.
- The impact of AI-generated diagnoses on clinical decision-making requires investigation.
- Corneal diseases necessitate accurate and timely diagnosis.
Purpose of the Study:
- To evaluate the influence of AI's diagnostic suggestions on ophthalmologists' accuracy.
- To determine if AI assistance impacts diagnostic accuracy differently based on specialist experience.
- To assess the reliability of AI in guiding corneal disease diagnosis.
Main Methods:
- A cross-sectional study involving 60 cases (30 infectious keratitis, 30 immunological keratitis).
- Ophthalmologists (7 corneal specialists, 16 non-specialists) assessed images with and without AI classification.
- AI diagnoses were intentionally incorrect in 30% of cases to test misleading guidance.
Main Results:
- Overall ophthalmologist accuracy did not significantly change post-AI assistance (75.2% vs. 75.9%, P=0.59).
- Corneal specialists' accuracy remained stable even with incorrect AI guidance (60.3% vs. 53.2%, P=0.11).
- Non-specialists' accuracy significantly decreased when misled by incorrect AI diagnoses (54.5% vs. 31.6%, P<0.001).
Conclusions:
- AI diagnostic support did not significantly alter overall ophthalmologist accuracy.
- Less experienced ophthalmologists are more susceptible to incorrect AI guidance than corneal specialists.
- Ophthalmologist experience remains a critical factor in accurate diagnosis, even with AI tools.
