Task design for crowdsourced glioma cell annotation in microscopy images

Area of Science:

• Computational pathology
• Machine learning
• Neuro-oncology

Background:

• Crowdsourcing is established for cell annotation in computational pathology.
• Glioma cell detection is challenging due to cell similarity and diffuse invasiveness.
• Multiplexed immunofluorescence microscopy is crucial for visualizing glioma microenvironments.

Purpose of the Study:

• To explore crowdsourcing for glioma cell detection.
• To develop and validate a crowdsourcing task for high-quality annotations.
• To train machine learning models for glioma cell identification.

Main Methods:

• Iterative task design and crowdworker training on Amazon Mechanical Turk.
• Majority or weighted voting for consensus annotation.
• Training YOLO convolutional neural network models with diverse image representations.

Main Results:

• 712 crowdworkers annotated 235 images, achieving a mean F1 score of 0.627.
• YOLOv8 models reached an average F1 score of 0.69.
• Demonstrated potential transferability to images without tumor markers, particularly in IDH-wildtype glioblastoma.

Conclusions:

• Crowdsourcing is feasible for generating labels for machine learning in glioma microenvironment analysis.
• This method provides a viable approach for tackling challenging annotation tasks in computational pathology.
• The trained models show promise for clinical applications in glioma research.