Automatic needle detection and real-time Bi-planar needle visualization during 3D ultrasound scanning of the liver

2D ultrasound (US) image guidance is used in minimally invasive procedures in the liver to visualize the target and the needle. Needle insertion using 2D ultrasound keeping the transducer position to view needle and reach target is challenging. Dedicated needle holders attached to the US transducer help to target in plane and at a specific angle. A drawback of this is that, the probe is fixed to the needle and cannot be rotated to assess the position of the needle in a perpendicular plane. In this study, we propose an automatic needle detection and tracking method using 3D US imaging to improve image guidance and visualization of the target in the liver with respect to the needle during these interventional procedures. The method utilizes a convolutional neural network for detection of the needle in 3D US images. In a subsequent step, the output of the convolutional neural network is used to detect needle candidates, which are fed into a final tracking step to determine the real needle position. The needle position is used to present two perpendicular cross-sectional planes of the 3D US image containing the needle in both directions. Performance of the method was evaluated in phantoms and in-vivo data by calculating the needle position distance and needle orientation angle between segmented needles and reference ground truth needles, which were manually annotated by an observer. The method successfully detects the needle position and orientation with mean errors of 1 mm and 2°, respectively. The proposed method yields a robust automatic needle detection and visualization at a frame rate of 3 Hz in 3D ultrasound imaging of the liver.