Calibration-Jitter: Augmentation of hyperspectral data for improved surgical scene segmentation

Area of Science:

• Medical Imaging
• Machine Learning
• Computer Vision

Background:

• Semantic surgical scene segmentation is vital for accurate tissue identification and delineation during surgery.
• Hyperspectral imaging (HSI) offers rich spectral information beyond visible light, enhancing tissue characterization.
• Current machine learning models struggle with variations in illumination and sensor sensitivity, limiting generalization.

Purpose of the Study:

• To introduce a novel spectral augmentation technique, Calibration-Jitter, for hyperspectral imaging in surgical applications.
• To improve the robustness and generalization of machine learning models for semantic surgical scene segmentation.
• To address the limitations of traditional augmentation methods in handling HSI variations.

Main Methods:

• Developed Calibration-Jitter, a spectral augmentation technique simulating hyperspectral calibration variations.
• Applied Calibration-Jitter to train a SegFormer model for semantic segmentation on a neurosurgical dataset.
• Evaluated model performance using F1-score, comparing against existing augmentation strategies.

Main Results:

• Calibration-Jitter significantly improved semantic segmentation performance on the neurosurgical dataset.
• The SegFormer model trained with Calibration-Jitter achieved a F1-score of 74.35%, outperforming the previous best of 70.2%.
• The technique demonstrated enhanced model generalization by effectively handling illumination and sensor sensitivity variations.

Conclusions:

• Calibration-Jitter is an effective spectral augmentation technique for improving hyperspectral imaging-based surgical scene segmentation.
• This method enhances the reliability of machine learning models in real-world surgical deployments.
• The advancement addresses critical limitations in training data augmentation for HSI analysis in medicine.