Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation.

Area of Science:

• Medical Imaging
• Computer Graphics
• Augmented Reality

Background:

• Autostereoscopic 3D image overlay is crucial for augmented reality (AR) surgical navigation.
• Existing methods lack high geometric fidelity and quantitative evaluation of 3D image accuracy.

Purpose of the Study:

• To propose a novel AR device for high-fidelity 3D image surgical overlay.
• To develop a GPU-based integral imaging pipeline and an automatic calibration paradigm for accurate 3D display.

Main Methods:

• Implemented a graphics processing unit (GPU)-based computer-generated integral imaging pipeline for real-time 3D display.
• Developed an automatic closed-loop 3D image calibration paradigm for undistorted 3D image display.
• Integrated a 3D display, AR window, stereo camera, and workstation into a novel AR device.

Main Results:

• Achieved rendering speeds of 50-60 fps for surface models and 5-8 fps for large medical volumes at 2560x1600 resolution.
• Demonstrated sub-millimeter geometric accuracy for undistorted 3D images post-calibration.
• Phantom experiments showed satisfactory image registration and overlay accuracy, confirming system usability in simulated oral and maxillofacial surgery.

Conclusions:

• The proposed GPU-based pipeline and calibration paradigm enable accurate and real-time autostereoscopic 3D display for AR surgical navigation.
• The novel AR device offers high geometric fidelity and sub-millimeter accuracy, significantly improving surgical overlay capabilities.
• The system's performance and usability were validated through rendering benchmarks and surgical simulation experiments.