Identifying Borderline Trachoma Grades Using a Three-Latent Class Model

The WHO has a simplified grading system for assessing trachoma. However, even for experts, it can be difficult to classify certain cases as strictly positive or negative for a given grade. Given the absence of a true gold standard, we performed a Latent Class Analysis (LCA) on a set of 200 graded photos of the superior tarsal conjunctiva. Ten trained graders assessed the presence of two trachoma grades: trachomatous inflammation-follicular (TF) and trachomatous inflammation-intense (TI). The LCA was modeled in two different ways: first with two classes (presence/absence), and then with three classes, with the extra class presumed to represent a more discrepant "borderline" case. Cohen's κ-statistics measuring agreement between graders were calculated for TF and TI grades (separately) before and after removing the third latent class. The κ-statistic increased by 0.10 (95% CI = 0.72-0.85; P <0.001) for TF and 0.13 (95% CI = 0.81-0.90; P <0.001) for TI, indicating that the third latent class represented a discrepant-case borderline class. The identification of borderline grading cases using a three-class LCA may be useful in creating balanced grader certification examinations that represent the full spectrum of disease. Additionally, a multiclass LCA could act as a probabilistic gold standard used to train and analyze future convolutional neural network models.