Learning from multiple annotators for medical image segmentation.

Area of Science:

• Medical Image Analysis
• Machine Learning
• Computer Vision

Background:

• Supervised machine learning for segmentation is sensitive to label quality.
• Medical image annotation suffers from high costs and inter-observer variability.
• Noisy labels limit the performance of automatic segmentation algorithms.

Purpose of the Study:

• To develop a method for jointly learning annotator reliability and expert consensus labels from noisy observations.
• To improve medical image segmentation accuracy despite label noise and disagreement.
• To address the challenge of high-cost, variable annotations in medical imaging.

Main Methods:

• Utilized two coupled Convolutional Neural Networks (CNNs).
• Jointly learned annotator reliability and consensus label distributions from noisy data alone.
• Employed a 'maximally unreliable' strategy to separate annotator behavior from consensus.
• Validated on MNIST, ISBI2015, BraTS, LIDC-IDRI, and a new QSMSC dataset.

Main Results:

• The proposed method consistently outperformed competing methods and baselines across all tested datasets.
• Performance gains were most significant with small numbers of annotations and high disagreement.
• The system effectively captured complex spatial characteristics of annotator errors.
• Demonstrated efficacy on diverse medical imaging tasks including multiple sclerosis lesions, brain tumors, and lung abnormalities.

Conclusions:

• The novel coupled CNN approach successfully addresses the challenge of noisy and variable labels in medical image segmentation.
• This method offers a robust solution for improving segmentation accuracy in data-scarce or high-disagreement scenarios.
• The ability to model annotator reliability enhances the trustworthiness and performance of automated segmentation systems.