Related Experiment Video
Updated: Aug 1, 2026
09:11
Automation of the Micronucleus Assay Using Imaging Flow Cytometry and Artificial Intelligence
Published on: January 27, 2023
2.2K
Annotation protocol and crowdsourcing multiple instance learning classification of skin histological images: The
Rocío Del Amor1, Jose Pérez-Cano2, Miguel López-Pérez2
1Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València, Valencia, Spain.
Artificial Intelligence in Medicine
|November 4, 2023
Summary
We developed a crowdsourcing method for labeling histopathology images, creating the CR-AI4SkIN dataset for cutaneous spindle cell neoplasms. This AI approach achieved a 0.79 F1 score, outperforming expert-labeled models.
Area of Science:
- Computational pathology and artificial intelligence in oncology.
- Development of novel machine learning algorithms for medical image analysis.
Background:
- Digital pathology (DP) and whole slide images (WSIs) are crucial for tumor diagnosis and prognosis.
- Artificial intelligence (AI) systems, particularly deep learning (DL), enhance diagnostic support but require extensive labeled data.
- Creating histopathology datasets is labor-intensive and time-consuming.
Purpose of the Study:
- To introduce a crowdsourcing-multiple instance labeling/learning protocol for efficient dataset creation.
- To develop and validate an AI-driven diagnostic tool for cutaneous spindle cell (CSC) neoplasms using limited expert labels.
- To establish the CR-AI4SkIN dataset, the first of its kind for CSC neoplasms.
Main Methods:
- Developed a crowdsourcing protocol to create the CR-AI4SkIN dataset (271 WSIs) with expert and non-expert labels.
- Implemented an automatic region of interest (ROI) extractor based on expert-annotated regions.
- Utilized contrastive learning for patch embedding and a Gaussian process classifier for crowdsourced WSI classification.
Main Results:
- The CR-AI4SkIN dataset contains 271 WSIs of 7 CSC neoplasms with multi-level annotations.
- The crowdsourcing-multiple instance learning method achieved an F1 score of 0.7911 on a binary classification task (malign vs. benign).
- This method outperformed traditional crowdsourcing aggregation techniques and a supervised model trained on expert labels (F1-score = 0.6035).
Conclusions:
- The proposed crowdsourcing multiple instance learning protocol is effective for annotating histopathology data.
- Automatic ROI extraction, contrastive embedding, and Gaussian process classification are viable for crowdsourced AI diagnostic tasks.
- This approach significantly reduces the reliance on extensive expert labeling for training AI in digital pathology.
