Multimodal Diagnostic Approach for Osteosarcoma and Bone Callus Using Hyperspectral Imaging and Deep Learning.

Area of Science:

• Biomedical Engineering
• Computational Pathology
• Medical Imaging

Background:

• Differentiating osteosarcoma from bone callus is difficult due to similar appearances.
• Current histopathological methods have limitations in distinguishing these conditions.

Purpose of the Study:

• To develop a multimodal deep learning framework (J-CAN) for accurate osteosarcoma classification.
• To integrate hyperspectral imaging (HSI) and H&E-stained pathology for enhanced diagnostic capabilities.

Main Methods:

• J-CAN framework combines MobileNetV2 for spatial features and 1D-CNN for spectral signatures from HSI (176 bands, 400-1000 nm).
• A self-attention mechanism was employed to prioritize critical spectral and spatial features.
• Performance was evaluated against conventional models like LSTM, SVM, and 1D-CNN.

Main Results:

• J-CAN achieved 87.33% accuracy, 89.07% sensitivity, and 85.49% specificity.
• The proposed framework demonstrated superior performance compared to traditional classification models.
• The integration of HSI and deep learning significantly improved classification accuracy.

Conclusions:

• HSI-driven deep learning offers a promising approach for automated and precise osteosarcoma diagnosis.
• J-CAN enhances diagnostic accuracy, aiding pathologists in differentiating osteosarcoma from bone callus.
• This technology has the potential to revolutionize clinical pathology workflows.