Do we need annotation experts? A case study in celiac disease classification.

Area of Science:

• Medical Imaging
• Computer-Aided Diagnosis
• Gastroenterology

Background:

• Clinical inference from medical images is crucial but relies on expert-labeled data.
• Acquiring expert-annotated datasets is time-consuming and costly.
• The necessity of expert knowledge for all medical imaging tasks is questioned.

Purpose of the Study:

• To investigate if non-expert annotations can replace expert knowledge in medical image analysis.
• To assess the impact of non-expert labeled data on model performance for celiac disease detection.
• To demonstrate that large non-expert datasets can overcome label noise challenges.

Main Methods:

• Utilized a database of endoscopy images for celiac disease assessment.
• Collected annotations from non-expert volunteers.
• Trained machine learning models using the non-expert annotated data.
• Focused on compensating for label noise through data volume, not algorithmic noise handling.

Main Results:

• Compelling empirical evidence shows non-expert annotations are viable.
• Sufficiently large datasets labeled by non-experts can compensate for label noise.
• Model performance was adequate despite the absence of expert-level annotation.

Conclusions:

• Expert knowledge may not be a prerequisite for certain medical imaging tasks.
• Large-scale, non-expert annotation can be a cost-effective alternative to expert labeling.
• This approach offers a scalable solution for building medical imaging datasets.