Optimization of Transfer Learning of Foundation Models for Hyperspectral Histologic Imaging

Area of Science:

• Computational pathology
• Digital pathology
• Medical imaging analysis

Background:

• Hyperspectral imaging (HSI) offers advanced tissue analysis but faces adoption challenges compared to traditional RGB imaging.
• Foundation models trained on RGB data hold potential for HSI analysis but require effective knowledge transfer strategies.

Purpose of the Study:

• To develop and evaluate techniques for transferring knowledge from RGB-trained histopathological foundation models to HSI data.
• To identify optimal hyperparameters for knowledge transfer in computational histopathology using HSI.

Main Methods:

• Fine-tuning three foundation models on a dataset of 89 whole-slide HSI images from 54 patients.
• Conducting a hyperparameter search to determine effective learning rates and weight decay for transfer learning.

Main Results:

• End-to-end fine-tuning demonstrated superior performance over other knowledge transfer methods.
• Low learning rates and high weight decay were identified as optimal hyperparameters for this transfer learning task.
• Findings challenge conventional approaches focusing solely on embedding layer training.

Conclusions:

• Effective techniques for adapting RGB-trained foundation models to HSI data in computational histopathology have been established.
• The study provides a practical framework for leveraging existing foundation models for advanced HSI-based digital pathology applications.