Related Experiment Videos
Lessons from a breast cell annotation competition series for school pupils
Wenqi Lu1, Islam M Miligy2,3, Fayyaz Minhas1
1Department of Computer Science, University of Warwick, Coventry, UK.
Scientific Reports
|May 13, 2022
Summary
School pupils participating in the "Beat the Pathologists" competition showed strong performance in annotating tumor cells for artificial intelligence (AI) development. While their accuracy varied, this highlights the potential of non-experts in AI annotation tasks.
Area of Science:
- Computational pathology
- Artificial intelligence in medicine
- Digital pathology education
Background:
- COVID-19 pandemic led to extended home-schooling for pupils.
- Need for engaging educational activities and contributions to scientific research.
- Growing demand for large-scale annotated datasets for AI development in pathology.
Purpose of the Study:
- To assess school pupils' ability to annotate cells in breast cancer images for AI.
- To evaluate pupil performance against expert pathologists and machine learning models.
- To explore the potential of citizen science in AI annotation for pathology.
Main Methods:
- Two editions of a web-based competition, 'Beat the Pathologists', were conducted for UK school pupils.
- Participants annotated four cell types on Ki67-stained breast cancer images across four complexity levels.
- Pupil annotations were compared against expert ground truth and performance of AlexNet and VGG16 neural networks.
Main Results:
- Pupils achieved high accuracy in tumor cell annotation (F1 score 0.81).
- Lower accuracy was observed for non-tumor cells (F1 scores 0.75 and 0.59).
- Neural networks, particularly VGG16, outperformed pupils in non-tumor cell detection (F1 > 0.70 overall, 0.92 for tumor cells).
Conclusions:
- Non-experts, like school pupils, can contribute to AI annotation with adequate training.
- Competitions can foster public interest in pathology and encourage wider participation in data labeling.
- This approach offers a scalable solution for generating annotated data crucial for AI algorithm advancement.