Estimating kappa from binocular data and comparing marginal probabilities

Area of Science:

• Biostatistics
• Ophthalmology
• Medical Statistics

Background:

• Accurate assessment of ocular abnormalities requires reliable grader agreement.
• Paired eye data (left and right) present unique statistical challenges due to potential correlations.
• Existing methods for analyzing grader agreement in paired body structures may not fully address these correlations.

Purpose of the Study:

• To propose an alternative statistical procedure for analyzing grader agreement in paired eye assessments.
• To account for the correlation between observations in the right and left eyes.
• To provide a generalized McNemar test for comparing positive judgment probabilities between two graders.

Main Methods:

• Utilizes existing formulas for calculating weighted kappa and its standard error.
• Extends the McNemar test to compare paired binary data from two graders.
• Applies the proposed methods to analyze agreement in classifying ocular abnormalities.

Main Results:

• The proposed procedure effectively incorporates the correlation between paired eye observations.
• The generalized McNemar test allows for a more robust comparison of grader judgments.
• The method provides a reliable framework for assessing inter-grader reliability in ophthalmological studies.

Conclusions:

• The developed statistical approach offers an improvement for analyzing grader agreement in paired ocular data.
• This method enhances the accuracy of statistical analysis in ophthalmological research involving paired assessments.
• The generalized McNemar test provides a valuable tool for comparing diagnostic judgments between graders.