False Negative Rates in Benign Thyroid Nodule Diagnosis: Machine Learning for Detecting Malignancy

Thyroid nodules are common; up to 67% of adults will show nodules on high-quality ultrasound, and 95% of these nodules are benign. FNA cytology is a crucial step in determining the risk of malignancy, and a false negative diagnosis at this stage delays cancer treatment. The purpose of this study is to develop a predictive model using machine learning which can identify false negative FNA results based on less-invasive clinical data.

We conducted a retrospective medical record review at one academic and one community center. Inclusion criteria were thyroid nodules evaluated by ultrasound and FNA with a Bethesda II (benign) result or malignancy detected on pathology or FNA. Linear, non-linear, and ensemble models were generated with scikit-learn using 10-fold cross validation with repetition and compared with AUROC. The classification task was the prediction of malignancy using information acquired from less-invasive ultrasound and FNA.

A total of 604 subjects met inclusion criteria; 38 were diagnosed with malignancy. Of all algorithms tested, a Random Forest method achieved the best AUROC (0.64) in separating benign and malignant nodules, though the improvement over other tested algorithms was not statistically significant.

A Random Forest model performed better than random chance using readily available data obtained via standard evaluation of thyroid nodules. The diagnostic probability threshold of this model can be varied to minimize false positives at the cost of increasing the number of false negatives. Future studies will prospectively evaluate the model's performance.