Deep Learning to Localize Photoacoustic Sources in Three Dimensions: Theory and Implementation

Area of Science:

• Medical Imaging
• Biomedical Engineering
• Artificial Intelligence

Background:

• Accurate surgical tool tip localization and tracking are crucial for effective surgical and interventional procedures.
• Current methods for tool tip localization using photoacoustic imaging are limited in their dimensional tracking capabilities.
• Deep learning on photoacoustic channel data offers potential for advanced tool tip localization.

Purpose of the Study:

• To develop and evaluate novel deep learning-based 3D photoacoustic point source localization systems.
• To extend previous object detection approaches and introduce an instance segmentation-based system for enhanced localization.
• To theoretically derive and experimentally validate relationships between point source locations, sound speeds, and photoacoustic waveform data.

Main Methods:

• Development of a deep learning-based 3D photoacoustic point source localization system using an object detection approach.
• Derivation of theoretical relationships for 3D point source localization using photoacoustic channel data.
• Implementation of a novel deep learning instance segmentation-based 3D point source localization system incorporating sound speed estimation.

Main Results:

• The object detection and instance segmentation systems achieved high F1 scores (up to 99.82%) across simulated, phantom, and ex vivo data.
• Low Euclidean localization errors (mean ± SD) of approximately 1.5 mm were achieved in all tested datasets.
• The instance segmentation system demonstrated accurate simultaneous estimation of sound speed with low absolute errors.

Conclusions:

• The developed deep learning systems show significant potential for precise 3D localization and tracking of surgical tool tips using photoacoustic imaging.
• These advancements can enhance the safety and efficacy of minimally invasive surgical and interventional procedures.
• The integration of sound speed estimation further refines the capabilities of photoacoustic-based surgical navigation.