Related Experiment Video
Updated: Jun 4, 2026
Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone
Published on: January 2, 2020
Marker-less Body Surface Registration with 3-Dimensional Imaging for Percutaneous Intervention with Smartphone
Laetitia Saccenti1, Nicole A Varble2, Olivia Pena3
1Center for Interventional Oncology, National Institutes of Health Clinical Center, Bethesda, Maryland; Henri Mondor's Institute of Biomedical Research - Inserm, Creteil, France.
Purpose:
To evaluate the feasibility and accuracy of surface tracking registration with a smartphone augmented reality (AR) guidance system for percutaneous needle insertion in phantoms and in vivo.
Materials And Methods:
An AR application for needle guidance was developed (Unity) using smartphone platform with an integrated needle guide (Verza Needle Guide; Civco, Coralville, Iowa). Automatic registration using body surface tracking based on deep learning (Model Target, Vuforia) obviated the need for fiducials with no additional sensors or hardware. Multiplanar computed tomography (CT) images were volumetrically rendered to enable direct overlay on the body without segmentation. Accuracy was assessed on an abdominal phantom with 8 operators of varying experience. An in vivo study was conducted in 3 swine (N = 15 targets), where embolization coils implanted in liver, kidney, and muscle served as targets. Needle tip-to-target distance and angular error were measured on postprocedural CT.
Results:
In phantom experiments, the median accuracy was 4.8 mm (interquartile range [IQR], 3.3-7.7 mm) with a median angular error of 2.2° (IQR, 1.4°-4.3°). In vivo, the mean accuracy was 8.9 mm (SD ± 4.3), and the mean angular error was 4.2° (SD ± 2.2). Accuracy varied by organ (P = .012), with best results in muscle (5.1 mm), followed by kidney (8.7 mm) and liver (13.0 mm). Surface tracking compensated for external body shift but not internal organ motion.
Conclusions:
Smartphone AR with automatic surface tracking enabled guidance for needle insertion in vivo without fiducial markers or manual segmentation. The technology is feasible for simplifying or supplementing interventional radiology workflows using AR.
