One model is all you need: Multi-task learning enables simultaneous histology image segmentation and classification.

Area of Science:

• Computational pathology
• Digital pathology
• Machine learning in medicine

Background:

• Deep learning significantly advances digitised pathology slide analysis.
• Current deep learning models are often single-task and require large datasets.
• Adapting models for multiple tasks and limited data presents scalability challenges.

Purpose of the Study:

• To present a multi-task learning approach for simultaneous segmentation and classification of tissue structures.
• To leverage data from multiple sources for improved model performance.
• To facilitate biomarker discovery and enable transfer learning for computational pathology.

Main Methods:

• Developed a multi-task deep learning model (Cerberus) for segmentation and classification.
• Trained the model on over 600,000 objects and 440,000 patches from diverse data sources.
• Aligned tasks by tissue type and resolution for simultaneous prediction within a single network.

Main Results:

• The Cerberus model successfully processed 599 colorectal whole-slide images, localizing millions of nuclei, glands, and lumina.
• Feature sharing improved performance on additional tasks like nuclear classification and signet ring cell detection via transfer learning.
• The approach demonstrated effective segmentation and classification of nuclei, glands, lumina, and tissue regions.

Conclusions:

• Multi-task learning with feature sharing enhances the performance and scalability of deep learning models in computational pathology.
• The developed resource and model address a key barrier in creating explainable AI for pathology.
• This work paves the way for more robust and versatile AI tools in digital pathology research.