Related Experiment Video
Updated: Jun 10, 2025
Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
Published on: August 9, 2024
Nested Semi-Transparent Isosurface Simulated Volume-Rendering (NESTIS-VR) - An efficient on-device rendering approach
Fabian N Necker1, Marc L Melcher2, Stephan Busque2
1Incubator for Medical Mixed Reality at Stanford (IMMERS), Department of Radiology, Stanford University, Palo Alto, CA, USA; Institute of Functional and Clinical Anatomy, Faculty of Medicine, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany; Department of Radiology, Stanford University, Palo Alto, CA, USA.
Background And Objective:
Volume-renderings of computed tomography or magnetic resonance angiograms (MRAs) are routinely used by surgeons in the preoperative assessment of vascular anatomy in kidney donors. Stereoscopic headsets (OST-HMD) like Microsoft HoloLens allow intuitive interaction with three-dimensional content for more intuitive comprehension, but do not allow real-time ray-casting volume-rendering of medical volume datasets on-device due to computational limitations.
Methods:
We introduce NEsted Semi-Transparent Isosurface Simulated Volume-Rendering (NESTIS-VR), as an on-device alternative to ray-casting volume-rendering and developed an application for HoloLens to render kidney donor MRAs with interactive control of fundamental rendering parameters. We compared NESTIS-VR with current standard pre-calculated 2D ray-cast volume-renderings in an observational study with 2 expert kidney transplant surgeons, measuring their confidence in pre-operatively assessing the kidney pedicle arterial anatomy in 20 potential donors. We also compared it against other 3D rendering techniques to understand which features contributed most to any improvements.
Results:
Real-time stereoscopic three-dimensional (3D) NESTIS-VR in Augmented Reality significantly improves surgeons' confidence compared with pre-calculated conventional two-dimensional (2D) ray-casting volume-rendered images (p = 0.0415/p = 0.00003). 2D non-stereoscopic NESTIS-VR was significantly superior to pre-calculated 2D ray-casting volume-rendered images for both surgeons (p = 0.044/p = 0.0003). Single isosurface 2D rendering was significantly superior than pre-calculated 2D volume-rendered images for one surgeon. There was no significant difference between binocular 3D display over 2D views with NESTIS-VR or between constrained and unconstrained vantage points for 2D viewing.
Conclusion:
NESTIS-VR provides a new approach to rendering medical datasets in computationally limited OST-HMD headsets and significantly increases surgeons' confidence of kidney donor arterial anatomy. The principal confidence benefit arises from providing surgeons interactive control over rendering parameters compared to pre-calculated renderings at preset parameters whilst rendering on-device and keeping the OST-HMD untethered from a workstation.
