Attention-aware fully convolutional neural network with convolutional long short-term memory network for...

Area of Science:

• Medical Imaging
• Radiation Oncology
• Artificial Intelligence

Background:

• Real-time motion management is crucial in radiation therapy (RT) for accurate tumor targeting.
• Ultrasound imaging offers advantages like high soft tissue contrast and real-time capability without ionizing radiation.
• Robotic-arm-mounted ultrasound systems are promising for motion-guided RT.

Purpose of the Study:

• To develop a novel deep learning-based real-time motion tracking strategy for ultrasound image-guided RT.
• To enhance the precision and efficiency of motion management during radiation delivery.

Main Methods:

• A deep learning model combining attention-aware fully convolutional neural network (FCNN) and convolutional long short-term memory (CLSTM) was developed.
• A glimpse sensor module within FCNN focused on regions of interest, enhancing feature extraction.
• Multitask loss strategy (bounding box, localization, saliency, adaptive weighting) facilitated training convergence.

Main Results:

• The system demonstrated a mean tracking error of 0.97 ± 0.52 mm and a maximum error of 1.94 mm for 85 landmarks across 39 cases.
• Tracking speed ranged from 66 to 101 frames per second per landmark on GPU, exceeding ultrasound imaging rates.
• The deep learning approach proved robust against ultrasound image artifacts like speckle noise.

Conclusions:

• The developed deep learning-based motion estimation strategy is accurate and robust for ultrasound-guided RT.
• The remarkable tracking speed is suitable for real-time applications in radiation therapy.
• This approach offers a valuable tool for real-time beam gating or multileaf collimator (MLC) tracking in RT.