Robust real-time segmentation of bio-morphological features in human cherenkov imaging during radiotherapy via deep...

Cherenkov imaging enables real-time visualization of megavoltage X-ray or electron beam delivery to the patient during radiation therapy (RT). Bio-morphological features, such as vasculature, seen in these images are patient-specific signatures that can be used for verification of positioning and motion management that are essential to precise RT treatment. However, no concerted analysis of this biological feature-based tracking has been utilized until now because of the slow speed and accuracy of conventional image processing for feature segmentation.

This study aims to demonstrate the first deep learning framework for such an application, achieving video frame rate processing.

To address the challenge of limited annotation of bio-morphological features in Cherenkov images, a transfer learning strategy was applied. A fundus photography dataset including 20,529 patch retina images with ground-truth vessel annotation was used to pre-train a ResNet based segmentation framework. Subsequently, a small Cherenkov dataset (1483 images from 212 treatment fractions of 19 breast cancer patients) with known annotated vasculature masks was used to fine-tune the model for accurate segmentation prediction.

The well-trained model was tested on clinical Cherenkov dataset which was not used in fine-tune steps. This deep learning framework achieved consistent and rapid segmentation of Cherenkov-imaged bio-morphological features on a test dataset containing 19 patients (179 images), including subcutaneous veins, scars, and pigmented skin. The average segmentation by the model achieved a Dice score of 0.85 and required less than 0.7 ms processing time per instance.

The model demonstrated outstanding consistency against input image variances and speed compared to conventional manual segmentation methods, laying the foundation for online segmentation in real-time monitoring in a prospective setting.