Test-time augmentation with synthetic data addresses distribution shifts in spectral imaging

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Technology

Background:

• Surgical scene segmentation is vital for context-aware surgical assistance.
• Hyperspectral imaging (HSI) offers advantages over RGB data for segmentation.
• Existing HSI datasets are limited and lack clinical tissue variation.

Purpose of the Study:

• To explore spectral imaging distribution shifts caused by organ perfusion alterations.
• To introduce a novel strategy to mitigate these distribution shifts using synthetic data.

Main Methods:

• Analysis of 615 hyperspectral images from 16 pigs with varying organ perfusion states.
• Development of a test-time augmentation strategy utilizing synthetic data.
• Evaluation of state-of-the-art segmentation networks under induced perfusion changes.

Main Results:

• Perfusion changes significantly impacted segmentation network performance, with up to a 93% decline for kidneys under ischemia.
• The proposed method demonstrated up to a 4.6-fold improvement over state-of-the-art approaches.
• Organ-specific variations in performance decline were observed due to perfusion alterations.

Conclusions:

• The developed approach enhances neural network generalization in spectral imaging.
• This method shows potential for improving surgical assistance across diverse pathologies.
• Addressing distribution shifts is crucial for robust hyperspectral imaging applications in surgery.