Foundation Model-Based Zero-Shot Tissue Segmentation of Pathological Images via the Mixture of Local-to-Global...

Area of Science:

• Digital pathology
• Computational oncology
• Medical image analysis

Background:

• Accurate tissue segmentation in pathological images is vital for cancer diagnosis and prognosis.
• Traditional models struggle with segmenting rare tissue types due to annotation challenges and zero-shot learning limitations.

Purpose of the Study:

• To develop a novel architecture, ZSPMLG, for accurate pixel-wise tissue segmentation in pathological images.
• To enable segmentation of both seen and unseen tissue types using text descriptions and a foundation model.

Main Methods:

• Utilized a pathology vision-language foundation model (CONCH) and large language models (LLMs) to generate text descriptions for tissue categories.
• Developed the ZSPMLG architecture incorporating Mixture of Local Experts (MoLE) and Mixture of Global Experts (MoGE) modules for multi-scale representation fusion.
• Employed a convolutional layer to map pixel-level representations to category prototypes for segmentation.

Main Results:

• The ZSPMLG architecture demonstrated superior performance in segmenting both seen and unseen tissue categories across three datasets.
• Successfully addressed the limitations of traditional models in handling tissue types with zero training samples.

Conclusions:

• The proposed ZSPMLG method offers a significant advancement in pathological image segmentation, particularly for rare tissue types.
• This approach enhances the potential for improved cancer diagnosis and prognosis through more comprehensive tissue analysis.