Enhancing Free-Hand 3-D Photoacoustic and Ultrasound Reconstruction Using Deep Learning.

Area of Science:

• Medical Imaging
• Biomedical Engineering
• Computer Vision

Background:

• Handheld photoacoustic and ultrasound (PAUS) imaging faces limitations in 3D reconstruction due to narrow field of view and motion estimation challenges.
• Accurate 3D reconstruction from sequential 2D images is crucial but difficult without external positional sensors.

Purpose of the Study:

• To introduce MoGLo-Net, a motion-based learning network with global-local self-attention, for enhanced 3D reconstruction in PAUS imaging.
• To improve motion parameter estimation for accurate 3D freehand ultrasound reconstruction.

Main Methods:

• Developed MoGLo-Net utilizing a global-local self-attention module to exploit critical image regions for motion estimation.
• Employed patch-wise correlation to generate motion-correlated volumes and a custom loss function for robust learning.
• Applied the network to B-mode, Doppler ultrasound, and photoacoustic imaging for 3D reconstruction.

Main Results:

• MoGLo-Net demonstrated superior quantitative and qualitative performance compared to state-of-the-art methods.
• Successfully enabled 3D visualization of vasculature using Doppler ultrasound and photoacoustic imaging.
• Achieved accurate motion estimation crucial for freehand 3D reconstruction without external sensors.

Conclusions:

• MoGLo-Net significantly advances 3D reconstruction capabilities in PAUS imaging.
• The method offers a robust solution for complex 3D structure visualization and vasculature imaging.
• Publicly available code facilitates further research and development in the field.