Manifold Embedding and Semantic Segmentation for Intraoperative Guidance With Hyperspectral Brain Imaging

Area of Science:

• Medical imaging
• Computational pathology
• Surgical oncology

Background:

• Hyperspectral imaging (HSI) offers non-invasive, real-time tissue characterization during surgery.
• High dimensionality of HSI data poses challenges for in vivo processing and accurate classification.
• Existing dimensionality reduction methods can be slow and inconsistent, impacting tissue classification.

Purpose of the Study:

• To introduce a novel dimensionality reduction scheme and processing pipeline for hyperspectral images.
• To enable detailed tumor classification maps for intra-operative margin definition in brain surgery.
• To overcome limitations of existing manifold embedding techniques in terms of speed and consistency.

Main Methods:

• A two-step framework combining an extended T-distributed stochastic neighbor embedding for dimensionality reduction.
• Application of a Semantic Texton Forest for semantic segmentation and tissue classification on reduced data.
• In vivo validation of the proposed framework in a clinical setting.

Main Results:

• The proposed method successfully reduces hyperspectral data dimensionality while preserving crucial information.
• Accurate tumor classification maps were generated, aiding in intra-operative margin assessment.
• The Semantic Texton Forest effectively classified tissue types based on the embedded hyperspectral data.

Conclusions:

• The developed framework offers a robust solution for real-time hyperspectral image processing in surgical environments.
• This approach demonstrates significant potential for improving tumor margin definition and patient outcomes in brain surgery.
• The system's in vivo validation highlights its clinical applicability and value.