Wide Inter-institutional Variation in Performance of a Molecular Classifier for Indeterminate Thyroid Nodules

The Afirma gene expression classifier (GEC) is used to assess malignancy risk in indeterminate thyroid nodules (ITNs) classified as Bethesda category III/IV. Our objective was to analyze GEC performance at two institutions with high thyroid cytopathology volumes but differing prevalence of malignancy.

Retrospective analysis of all ITNs evaluated with the GEC at Memorial Sloan Kettering Cancer Center (MSK; n = 94) and Mount Sinai Beth Israel (MSBI; n = 71). These institutions have differing prevalences of malignancy in ITNs: 30-38 % (MSK) and 10-19 % (MSBI). Surgical pathology was correlated with GEC findings for each matched nodule. Performance characteristics were estimated using Bayes Theorem.

Patient and nodule characteristics were similar at MSK and MSBI. The GEC-benign call rates were 38.3 % (MSK) and 52.1 % (MSBI). Of the GEC-benign nodules, 8.3 % (MSK) and 13.5 % (MSBI) were treated surgically. Surgical pathology indicated that all of GEC-benign nodules were benign. Of the GEC-suspicious nodules, 60.0 % (MSK) and 61.7 % (MSBI) underwent surgery. Positive predictive values (PPVs) for GEC-suspicious results were 57.1 % (95 % CI 41.0-72.3) at MSK and 14.3 % (95 % CI 0.2-30.2) at MSBI. The estimated negative predictive values (NPVs) were 86-92 % at MSK and 95-98 % at MSBI.

There were wide variations in the Afirma GEC-benign call rate, PPV, and NPV between MSBI (a comprehensive health system) and MSK (a tertiary referral cancer center), which had differing rates of malignancy in ITNs. The GEC could not routinely alter management in either institution. We believe that this assay would be expected to be most informative in practice settings where the prevalence of malignancy is 15-21 %, such that NPV >95 % and PPV >25 % would be anticipated. Knowing the prevalence of malignancy in ITNs at a particular institution is critical for reliable interpretation of GEC results.