Is Caruncular Oedema a Reliable Indicator in the Clinical Activity Score for Thyroid Eye Disease? An Analysis of Bias
Marie Callet1, Laura Licini2, Océanne Haelewyn1
1Department of Oculoplastics and Ophthalmology, Rothschild Foundation Hospital, Paris, France.
Objective:
To evaluate the inter-observer reliability of the caruncular oedema criterion within the Clinical Activity Score (CAS) for thyroid eye disease (TED), and to investigate potential biases affecting its grading, including the influence of image review order and confounding from conjunctival hyperaemia and caruncular erythema.
Design And Patients:
We retrospectively reviewed 773 standardised photographs from 261 TED patients.
Measurements:
Three independent ophthalmologists graded each image for caruncular oedema, caruncular erythema, and conjunctival hyperaemia. To assess the effect of repetition bias, a subset of images was evaluated in both random and sequential (chronological) order. Inter-observer agreement was quantified using Cohen's kappa, systematic differences were tested with McNemar's test, and agreement strength was interpreted according to Landis and Koch's classification.
Results:
Inter-rater agreement for caruncular oedema was slight (κ = 0.10, 95% CI: 0.06-0.15), and substantially lower than for conjunctival hyperaemia (κ = 0.30, 95% CI: 0.26-0.35). When comparing sequential with random readings, agreement was poor for two graders, both demonstrating significant order-related bias (p = 0.022 and < 0.001), and only moderate but non-significant for the third (κ = 0.49, p = 0.147). Across graders, oedema was more often identified in sequential order, consistent with a repetition ("yes-set") bias. Furthermore, caruncular erythema and conjunctival hyperaemia were strongly associated with oedema scoring: a red or congested caruncle markedly increased the likelihood of false-positive oedema calls. These confounders likely explain the limited reproducibility of this CAS item.
Conclusions:
Caruncular oedema scoring within the CAS shows poor inter-observer reliability and vulnerability to cognitive and visual biases. Conjunctival and caruncular hyperaemia act as significant confounders, inflating disease activity scores and potentially shifting patients across the CAS threshold (> 3) that guides immunosuppressive treatment. Given these limitations, refinement or replacement of this criterion, through either objective imaging tools or alternative markers such as ocular hypertension, should be considered to improve the reproducibility and diagnostic value of the CAS in both ophthalmology and endocrinology practice.
