How reliable is the lacrimal scintigraphy report? An inter-observer agreement and reliability study

To assess the inter-observer agreement and reliability as well as intra-observer repeatability for lacrimal scintigraphy (LS) reports with and without considering the irrigation test results.

A prospective, observational, cross sectional study. Two masked clinicians (lacrimal surgeon and nuclear medicine specialist) independently reported 100 LS images (50 patients of >6 years of age with unilateral anophthalmic socket) in a university hospital. The lacrimal surgeon performed a diagnostic irrigation test and repeated the report of the same LS images 2y after the first report (intra-observer agreement). A weighted Kappa analysis was performed to determine inter-observer agreement and reliability as well as intra-observer repeatability for the type (normal, partial and complete obstruction) and location (presac, preduct, and intraduct) of the obstruction. Subgroup analysis was also performed with consideration of irrigation test results.

A significantly moderate agreement was found between lacrimal surgeon and nuclear medicine specialist for both the type (Kappa=0.55) and location (Kappa=0.48) of obstruction. Agreement values were higher for the type (Kappa=0.61 vs 0.41) and location (Kappa=0.56 vs 0.31) of obstruction in cases with normal than abnormal irrigation test. Strong and significant intra-observer (lacrimal surgeon) repeatability was found for both the type (Kappa=0.66) and location (Kappa=0.69) of obstruction. LS showed no to slight reliability based on irrigation test.

A moderate agreement is found between lacrimal surgeon and nuclear medicine specialist regarding the interpretation of LS suggesting the importance of consensus groups among nuclear medicine specialists and lacrimal surgeons to create a common language for interpretation of LS. Intra-observer repeatability is strong for the lacrimal surgeon.