Related Experiment Video
Updated: May 28, 2026
Automatic Surgery in Transcatheter Aortic Valve Replacement Using Augmented Reality
Published on: August 9, 2024
Augmented reality visualization during laparoscopic radical prostatectomy
Tobias Simpfendörfer1, Matthias Baumhauer, Michael Müller
1Department of Urology, SLK-Kliniken Heilbronn, University of Heidelberg, Heilbronn, Germany. tobias.simpfendoerfer@med.uni-heidelberg.de
Purpose:
We present an augmented reality (AR) navigation system that conveys virtual organ models generated from transrectal ultrasonography (TRUS) onto a real laparoscopic video during radical prostatectomy. By providing this additional information about the actual anatomy, we can support surgeons in their working decisions. This work reports the system's first in-vivo application.
Materials And Methods:
The system uses custom-developed needles with colored heads that are inserted into the prostate as soon as the organ surface is uncovered. These navigation aids are once segmented in three-dimensional (3D) TRUS data that is acquired right after the placement of the needles and then continuously tracked in the laparoscopic video images by the surgical navigation system. The navigation system traces the navigation aids in real time and computes a registration between TRUS image and laparoscopic video based on the two-dimensional-three dimensional (2D-3D) point correspondences. With this registration, the system correctly superimposes TRUS-based 3D information on an additional AR monitor placed next to the normal laparoscopic screen. Surgical navigation guidance took place until the prostate was removed from the rectal wall. Finally, the navigation aids were removed together with the specimen inside the specimen bag.
Results:
The initial human in-vivo application of the surgical navigation system was successful. No complications occurred, the prostate was removed together with the navigation aids, and the system supported the surgeons as intended with an AR visualization in real time. In case of tissue deformations, changes in the spatial configuration of the navigation aids are detected, which preserves the system from erroneous navigation visualization.
Conclusions:
Feasibility of the navigation system was shown in the first in-vivo application. TRUS information could be superimposed via AR in real time. To show the benefit for the patient, results obtained from a larger number of trials are needed.
