Effects of examiner competency on the amblyopia screening performance of three visual acuity tests through Monte
Insights
Automating amblyopia screening requires understanding examiner error. This study simulated three tests, finding Keeler Acuity Cards for Infants (KACI) performed best, with Cardiff Acuity Targets (CAT) most affected by examiner competency.
Area of Science:
- Ophthalmology
- Pediatric Vision
- Medical Technology
Background:
- Amblyopia is a leading cause of vision loss in children, treatable if detected early.
- Current infant amblyopia screening relies on examiner expertise, introducing potential errors.
- Automating screening tests is crucial for accurate and accessible diagnosis.
Purpose of the Study:
- To evaluate the impact of examiner error rates on the performance of three preferential-looking amblyopia screening tests.
- To determine the relationship between examiner competency and the clinical effectiveness of KACI, CAT, and TAC.
- To provide foundational data for the automation of amblyopia screening protocols.
Main Methods:
- Simulated screening tests assessing visual acuity differences between eyes.
- Varied examiner error rates from perfect to random performance.
- Analyzed accuracy, precision, recall, False Negative Rate (FNR), and F1 score for KACI, CAT, and TAC.
Main Results:
- All three preferential-looking protocols showed performance dependency on examiner competency.
- Cardiff Acuity Targets (CAT) demonstrated the highest dependency due to numerous target presentations.
- Keeler Acuity Cards for Infants (KACI) exhibited the best overall performance, particularly high recall.
Conclusions:
- Examiner competency significantly influences the clinical performance of infant amblyopia screening tests.
- KACI is the most robust protocol, offering high recall essential for effective screening.
- Understanding these dependencies is vital for developing reliable automated amblyopia screening tools.
Abstract:
Amblyopia is amongst the leading causes of vision loss in children. This condition is reversible if diagnosed and treated early. Ideally, treatment should be instituted in early infancy, as the rate of success of treatment significantly drops after the age of seven. Screening for amblyopia in infants critically relies on examiner expertise, lending itself to errors. Automation and digitization of amblyopia screening tests is being actively pursued by the scientific community. Yet, for the known amblyopia screening protocols, little is known about the dependence of the relative performance on the error rate of an examiner. Yet, this information is key in designing an automated test. In the present paper we address the relationship between the examiner error rate (examiner competency) and the clinical performance of three major preferential-looking tests to screen for amblyopia, namely, Keeler Acuity Cards for Infants (KACI), Cardiff Acuity Targets (CAT), and Teller Acuity Cards (TAC). In our study, screening tests of the difference between visual acuity of both eyes were simulated, with the examiner error rate ranging from perfect testing to randomness. Amblyopia was defined as a difference of 0.2 logMAR between the eyes. The accuracy, precision, recall, False Negative Rate (FNR) and F1 score were determined as a function of the examiner error rate, showing that all three protocols are dependent on the examiner competency, with CAT being the most dependent, due to the higher number of target presentations within a test. The best performing protocol was KACI, which exhibited high recall, important for screening purposes.Clinical Relevance- This paper assesses the relative effects of examiner-related clinical performance of three major preferential-looking protocols for screening for amblyopia, which is foundational to automating these tests.
