Multiple-Instance Learning for thyroid gland disease classification: A hands-on experience

Area of Science:

• Digital pathology
• Computational diagnostics
• Artificial intelligence in medicine

Background:

• Morphological diagnosis of thyroid neoplasms is time-consuming and requires specialized expertise.
• Current artificial intelligence (AI) research often relies on meticulously curated datasets, which are costly and complex to prepare.
• Weakly-annotated, real-world data presents an alternative for AI model development in diagnostics.

Purpose of the Study:

• To develop and evaluate machine learning models using weakly-annotated, real-world data for thyroid gland neoplasm diagnosis.
• To assess the performance of a Multiple-Instance Learning (MIL) model trained on patient-level annotations.
• To identify potential data quality issues affecting AI model performance and compare annotation strategies.

Main Methods:

• Development of a Multiple-Instance Learning (MIL) model trained on patient-level annotations from 1102 patients and 5104 whole slide images (WSIs).
• Utilized "real-world" data without selective preprocessing.
• Compared classification accuracy using detailed slide-level versus coarse patient-level annotations on a smaller dataset (36 cases, 91 WSIs).

Main Results:

• The MIL model achieved an average test set F1-Score of 0.85 (±0.05) in discriminating benign from malignant thyroid conditions at the patient level.
• This represents the first reported AI model trained on patient-level data without prior labeling refinement.
• Identified data quality pitfalls, such as resection margin dye, that can lead to model overfitting.
• Detailed annotations significantly improved classification performance on smaller datasets.

Conclusions:

• AI models can effectively diagnose thyroid neoplasms using weakly-annotated, real-world data, reducing the need for extensive manual annotation.
• Patient-level annotations are feasible for training diagnostic AI models, though data quality must be carefully managed.
• The findings highlight the potential of AI to streamline pathological workflows and improve diagnostic efficiency.