Multimodal HSI-MRI system for tumor delineation in neurosurgical guidance

This study proposes the fusion of hyperspectral imaging (HSI) and magnetic resonance imaging (MRI) for tumor delineation in image-guided neurosurgery, comparing three clinically inspired registration strategies and presenting a novel visualization for tumor resection guidance from both imaging modalities under controlled phantom conditions.

A multimodal brain phantom dataset acquired with a 9.4T preclinical MRI scanner and an HSI camera was used. MRI tissues were segmented using a Segment Anything Model, while HSI data were classified with supervised XGBoost, SVM, and RF models. Three registration strategies were evaluated: surface-based registration, rigid landmark-based registration, and ArUco marker-based registration. Accuracy was assessed using fiducial registration error (FRE) and target registration error (TRE), and the overlap between modalities was quantified by the Dice similarity coefficient (DICE) and average surface distance (ASD).

Rigid landmark-based registration achieved the best performance (FRE of 1.7 mm, DICE of 85.4%), closely followed by the ArUco-based method, which required minimal infrastructure. Surface-based registration showed higher variability due to manual point acquisition. Overall, a high overlap between MRI and HSI tumor segmentations was observed and further enriching the information by integrating both approaches CONCLUSIONS: The proposed framework enables a reproducible evaluation of HSI-MRI fusion strategies and highlights the complementary value of volumetric MRI and surface-based HSI. Future work will focus on multimodal tissue classification, improved phantom designs, and validation with patient data.