Augmented Reality during Open Liver Surgery Using a Markerless Non-rigid Registration System.

Area of Science:

• Medical imaging
• Surgical technology
• Computer-assisted surgery

Background:

• Liver resection navigation is challenging due to complex, patient-specific anatomy and potential for large parenchymal deformations.
• Augmented reality (AR) offers potential for intraoperative guidance in hepatectomies but faces deformation challenges.

Purpose of the Study:

• To evaluate the clinical feasibility and anatomical accuracy of a novel vision-based AR system for open liver surgery.
• To assess the system's ability to guide hepatectomies and optimize resection margins despite significant liver deformations.

Main Methods:

• A non-rigid registration method using a physics-based elastic liver model and real-time finite element method was employed.
• A single RGB-D camera provided data to fit the model to actual liver deformations.
• In vivo AR was performed during hepatectomy in 4 patients, and an ex vivo experiment with 1 explanted liver was conducted for error quantification.

Main Results:

• The vision-based AR system demonstrated successful in vivo application with a fast setup (<10 min) and real-time virtual anatomy overlay.
• Ex vivo experiments quantified a root mean square error of 7.9 mm for internal landmark registration.
• The markerless AR system provided satisfactory 3D accuracy with minimal equipment and setup time.

Conclusions:

• The developed markerless AR system shows promising results for intraoperative navigation in liver resections.
• Further validation in larger prospective studies is necessary to confirm its impact on pathological margins and oncological outcomes.