Efficient WSI classification with sequence reduction and transformers pretrained on text.

Area of Science:

• Computational pathology
• Digital pathology
• Machine learning

Background:

• Language Models (LMs) demonstrate strong performance in diverse sequential data tasks.
• Whole Slide Image (WSI) analysis in digital pathology is suitable for transformer architectures.
• Large WSIs pose computational challenges for deep transformer models.

Purpose of the Study:

• To develop a method for classifying WSIs using deep transformer models.
• To address the computational burden of processing large WSIs.
• To leverage knowledge from text-pre-trained transformers for pathology tasks.

Main Methods:

• Introduction of SeqShort, a multi-head attention-based sequence shortening layer.
• Summarizing large WSIs into fixed-size sequences of feature vectors by removing redundant visual information.
• Applying SeqShort to text pre-trained transformer models for WSI classification.

Main Results:

• SeqShort effectively reduces computational costs for self-attention on large inputs.
• The method enables the inclusion of positional encodings for unordered image patches.
• Accurate WSI classification was achieved across different digital pathology tasks.
• Minimal fine-tuning (less than 0.1% of parameters) demonstrated effective knowledge transfer.

Conclusions:

• SeqShort facilitates the use of deep transformer models for WSI classification.
• Knowledge transfer from natural language processing to digital pathology is highly effective.
• This approach offers an efficient solution for large-scale WSI analysis.