Performance Comparison of Multi-Modal Fusion Techniques in Tissue Perfusion Analysis Using Homography Calibration

Area of Science:

• Biomedical Engineering
• Medical Imaging Analysis
• Computational Biology

Background:

• Accurate tissue perfusion analysis is crucial for non-invasive monitoring of physiological conditions.
• Existing multi-modal imaging systems require optimized data fusion and calibration for improved performance.
• The TTPD dataset provides a foundation for evaluating advanced image processing techniques.

Purpose of the Study:

• To investigate the impact of fusion techniques and camera calibration on multi-modal tissue perfusion analysis.
• To optimize data alignment and fusion strategies for enhanced classification accuracy.
• To identify the most effective combination of imaging modalities and fusion methods.

Main Methods:

• Utilized Homography-based calibration to reduce spatial discrepancies between infrared (IR), thermal, and RGB camera data.
• Applied early and late fusion approaches using deep learning models (ResNet50, ResNet101) for data integration.
• Evaluated classification performance based on different fusion strategies and modality combinations.

Main Results:

• Homography-based calibration significantly improved modality alignment and overall results.
• Late fusion strategies outperformed early fusion in classification accuracy.
• The combination of thermal and RGB modalities with late fusion yielded the highest performance.

Conclusions:

• Precise camera calibration is essential for accurate multi-modal tissue perfusion analysis.
• Late fusion, especially with thermal and RGB data, is a highly effective strategy.
• These findings contribute to developing robust, non-invasive tissue perfusion monitoring systems.