Multiple-Instance Learning for thyroid gland disease classification: A hands-on experience

The morphological diagnosis of thyroid gland neoplasms presents a dual challenge: it requires the expertise of highly trained specialists and considerable time, particularly when evaluating multiple whole slide images (WSIs) from a single patient. The integration of artificial intelligence (AI) techniques into the diagnostic workflow is a hot area of research. However, most studies rely on meticulously curated datasets, the preparation of which is both costly and fraught with complexities. This paper investigates the development of machine learning models using weakly-annotated "real-world" data, devoid of the selective preprocessing typical in common research datasets. Our study demonstrates that a Multiple-Instance Learning (MIL) model, trained on a weak patient-level annotations of 1102 patients encompassing 5104 WSIs, successfully discriminates between benign and malignant conditions at the patient level, achieving an average test set F1-Score of 0.85 with a standard deviation of 0.05. This study is, to our knowledge, the first to report findings from an AI model trained on patient-level data without prior labeling refinement. Additionally, we identify potential pitfalls in data quality that could induce model overfitting, such as the inadvertent inclusion of dye used to highlight resection margins, which correlates with the target variable. We also assessed the impact of detailed slide-level versus coarse patient-level annotations on classification accuracy using a smaller, more precisely annotated dataset of 36 laboratory cases (91 WSIs). The results indicate that detailed annotations substantially enhance classification performance in smaller datasets.