Related Experiment Video
Updated: Aug 16, 2026
Using an Automated Hirschberg Test App to Evaluate Ocular Alignment
Published on: March 24, 2020
Screening for amblyopia in preverbal children with photoscreening photographs: IV. Interobserver variability in
Alan H Kim1, Jessica Chen, Wanda Ottar-Pfeifer
1Wilmer Opthamological Institute, Johns Hopkins University of Baltimore, MD, USA.
Background:
Earlier detection of childhood vision disorders is a prominent goal of vision screening. The Medical Technology and Innovation (MTI) PhotoScreener addresses this objective. Use of this camera does not require verbal feedback and may be administered early in a child's development. Decreasing the variability in photograph grading results will boost the utility of any photoscreening system. This report aims to understand and to decrease intra- and inter-observer variability in grading photoscreening photographs.
Methods:
We dissected the photograph grading process and quantified the intra- and inter-observer agreement using intraclass plot correlation coefficients. We evaluated the outcome of a two grader verification system vs. adjudicated measurements with Receiver Operator Characteristic (ROC) curves.
Participants:
Data on 955 children under 5 years of age, normal except for refractive error, each with complete photoscreening and eye examination data, culled from two previous studies.
Main Outcome Measures:
Intra- and inter-observer agreement in measuring bright crescent dimensions and pupillary diameters. Sensitivity and specificity of detection of hyperopia.
Results:
Measurements of bright crescents are associated with greater variability than are measurements of pupillary diameters. Recognition and omission of light "rim" measurements from photograph grading will result in superior inter-observer agreement. Photograph independent errors increase variability and may be corrected by remeasurement. A verification system in which the most discrepant 5% of measurements are redone results in ROC curves similar to adjudicated dimension.
Conclusions:
We conclude: 1) two novices grading photographs can do as well as one expert in most cases; 2) the proposed grading methodology has undergone statistical validation and can be used in other areas of ophthalmology and medicine; and 3) inter-observer variability, one of the limitations of photoscreening photograph grading, can be reduced. For 95% of the photographs, two novices achieve similar true positive and true negative values with or without adjudication.
