Deep Learning of Spatiotemporal Filtering for Fast Super-Resolution Ultrasound Imaging

Super-resolution ultrasound (SR-US) imaging is a new technique that breaks the diffraction limit and allows visualization of microvascular structures down to tens of micrometers. The image processing methods for the spatiotemporal filtering needed in SR-US, such as singular value decomposition (SVD), are computationally burdensome and performed offline. Deep learning has been applied to many biomedical imaging problems, and trained neural networks have been shown to process an image in milliseconds. The goal of this study was to evaluate the effectiveness of deep learning to realize a spatiotemporal filter in the context of SR-US processing. A 3-D convolutional neural network (3DCNN) was trained on in vitro and in vivo data sets using SVD as ground truth in tissue clutter reduction. In vitro data were obtained from a tissue-mimicking flow phantom, and in vivo data were collected from murine tumors of breast cancer. Three training techniques were studied: training with in vitro data sets, training with in vivo data sets, and transfer learning with initial training on in vitro data sets followed by fine-tuning with in vivo data sets. The neural network trained with in vitro data sets followed by fine-tuning with in vivo data sets had the highest accuracy at 88.0%. The SR-US images produced with deep learning allowed visualization of vessels as small as [Formula: see text] in diameter, which is below the diffraction limit (wavelength of [Formula: see text] at 14 MHz). The performance of the 3DCNN was encouraging for real-time SR-US imaging with an average processing frame rate for in vivo data of 51 Hz with GPU acceleration.