Multimodal imaging platform for enhanced tumor resection in neurosurgery: integrating hyperspectral and pCLE...

This work presents a novel multimodal imaging platform that integrates hyperspectral imaging (HSI) and probe-based confocal laser endomicroscopy (pCLE) for improved brain tumor identification during neurosurgery. By combining these two modalities, we aim to enhance surgical navigation, addressing the limitations of using each modality when used independently.

We developed a multimodal imaging platform that integrates HSI and pCLE within an operating microscope setup using computer vision techniques. The system combines real-time, high-resolution HSI for macroscopic tissue analysis with pCLE for cellular-level imaging. The predictions of each modality made using Machine Learning methods are combined to improve tumor identification.

Our evaluation of the multimodal system revealed low spatial error, with minimal reprojection discrepancies, ensuring precise alignment between the HSI and pCLE. This combined imaging approach together with our multimodal tissue characterization algorithm significantly improves tumor identification, yielding higher Dice and Recall scores compared to using HSI or pCLE individually.

Our multimodal imaging platform represents a crucial first step toward enhancing tumor identification by combining HSI and pCLE modalities for the first time. We highlight improvements in metrics such as the Dice score and Recall, underscoring the potential for further advancements in this area.