Related Experiment Video
Updated: Jun 11, 2026
Quantitative Visualization and Detection of Skin Cancer Using Dynamic Thermal Imaging
Published on: May 5, 2011
A heatmap-based deep learning framework for multi-modal registration of VIS, NIR, and thermal images in
Maria Oniga1, Paul Florin Rus1, Rvazvan Condorovici1
1Department of Applied Electronics and Information Engineering, National University of Science and Technology POLITEHNICA Bucharest, Bucharest, Romania.
Introduction:
Multi-modal image registration leverages complementary information from diverse imaging sources to achieve precise spatial alignment. However, aligning visible (VIS), near-infrared (NIR), and thermal (TH) modalities remains challenging due to appearance differences and limited annotated datasets.
Methods:
This study proposes a ResU-Net-inspired framework combining heatmap prediction and homography estimation to enable joint NIR-TH registration via shared feature representations. A proprietary dataset of 155 VIS-NIR-TH skin lesion triplets was created from 62 patients, resulting in 465 images in total. Diagnoses were confirmed by dermatologists or histopathology. Lesions were distributed across the face, trunk, and limbs to evaluate registration robustness under spatial variability.
Results And Discussion:
Experiments showed that NIR-VIS registration consistently outperformed TH-VIS registration, reflecting NIR's richer structural content and higher spectral similarity with VIS. Despite limitations related to dataset size and acquisition variability, the framework demonstrated the feasibility of VIS-NIR-TH triplet registration and provides the first documented dataset of its kind for multi-modal skin lesion imaging.
