Observer variation and reproducibility of endoscopic ultrasonography

To investigate interobserver variation and reproducibility of endosonographic findings, both experienced and inexperienced endosonographers evaluated depth of tumor invasion (T stage) and presence of lymph node metastasis (N stage) in 50 patients with nonobstructing esophageal carcinoma. Results were compared with the findings by surgical pathology of the resected specimens. The kappa statistic (kappa) was used to assess interobserver and intraobserver agreement and consistency of accurate interpretation (reproducibility) for the two groups. Agreement between the experienced endosonographers was excellent (kappa = > .75) for T1 and T4 lesions, good (kappa = .61) for T3 lesions, but only poor (kappa = > .46) for T2 lesions. The overall agreement between the experienced endosonographers was equally good for both T and N stages. Agreement between the inexperienced endosonographers was poor for all T stages but was good for lymph node metastasis (kappa = .52). For experienced endosonographers, endosonographic reproducibility of histologically confirmed T4 lesions was excellent, followed closely by T3 and T2 lesions; T1 tumors were frequently interpreted differently by the same endosonographer. Reproducibility of N stage determinations was excellent for N0 lymph nodes and good for N1 nodes. Thus, for experienced endosonographers, interobserver agreement was excellent for all T stages except T2, whereas reproducibility of determination of depth of tumor invasion was good to excellent for T2, T3, and T4 lesions but poor for T1 lesions. As yet poorly defined operator and machine-dependent factors that cause misinterpretation of T1 and T2 tumors will require additional study.