Deep learning with mixed supervision for brain tumor segmentation | JoVE Visualize

Area of Science:

Medical Imaging
Artificial Intelligence
Computational Pathology

Background:

Current tumor segmentation methods rely heavily on fully annotated images, which are expensive and time-consuming to create due to the need for expert medical knowledge.
Weakly annotated images, providing only global labels (tumor presence/absence), are substantially cheaper but less informative for training segmentation models.
Bridging the gap between data availability and model performance is crucial for advancing automated medical image analysis.

Purpose of the Study:

To develop a deep learning model for tumor segmentation that effectively utilizes both fully annotated and weakly annotated training data.
To enhance segmentation accuracy by integrating image-level classification into standard segmentation networks.
To investigate the impact of varying ratios of weakly annotated to fully annotated data on segmentation performance.

Main Methods:

Proposed a deep learning architecture extending existing segmentation networks with an additional image-level classification branch.
Implemented a joint training strategy for both segmentation and classification tasks to leverage weakly annotated images.
Evaluated the method on brain tumor segmentation using magnetic resonance images from the Brain Tumor Segmentation 2018 Challenge dataset.

Main Results:

The proposed joint training approach significantly improved tumor segmentation performance compared to standard supervised learning methods.
The performance gains were directly proportional to the proportion of weakly annotated images included in the training set.
The integrated classification branch effectively guided the network to learn relevant features for segmentation, mitigating the risk of learning irrelevant patterns.

Conclusions:

Combining fully and weakly annotated data through a joint segmentation-classification deep learning model is a viable and effective strategy for tumor segmentation.
This approach offers a significant improvement in segmentation accuracy, particularly when a large amount of cost-effective, weakly labeled data is available.
The method holds promise for reducing the reliance on expensive manual annotations in medical image analysis tasks.