Northern Italy in the American South: Assessing interobserver reliability within the Milan System for Reporting...

The Milan System for Reporting Salivary Gland Cytopathology (MSRSGC) has been proposed to standardize salivary gland fine-needle aspiration (FNA) diagnoses. This study assessed salivary gland FNA results and risk of malignancy (ROM) rates at the University of North Carolina as well as the interobserver reliability (IOR) of the atypia of undetermined significance (AUS) and salivary gland neoplasm of uncertain malignant potential (SUMP) categories.

The electronic medical record was searched for FNA cases from 2010 to 2017 with subsequent surgical resections. Histologic diagnosis was used for gold-standard comparison. The original cytologic results were then converted into MSRSGC categories (nondiagnostic, nonneoplastic, AUS, benign neoplasm, SUMP, suspicious, and malignant). For the assessment of IOR, 23 cases were selected with enrichment for cases diagnosed as AUS (n = 11) or SUMP (n = 9). Six boarded cytopathologists and 1 cytopathology fellow assessed representative slides and provided an MSRSGC diagnosis for each case. Fleiss' κ coefficients were calculated to determine IOR.

The ROM was 33% for both AUS and SUMP cases; however, the risk of neoplasia was 56% for AUS cases and 100% for SUMP cases. Fleiss' κ for the AUS category was 0.217 (P < .05), and Fleiss' κ for the SUMP category was 0.024 (P = .74).

In this study assessing the IOR of MSRSGC categories, fair agreement and slight agreement were found for the AUS and SUMP categories, respectively. Observers preferentially used the AUS or benign neoplasm category for SUMP cases, perhaps because of unfamiliarity with SUMP as a diagnostic option. The initial adoption of a new reporting system will require a quality assessment to ensure that the system is reliable and useful for clinicians. Cancer Cytopathol 2018;126:390-6. © 2018 American Cancer Society.