A novel augmented reality to visualize the hidden organs and internal structure in surgeries

Area of Science:

• Medical Imaging
• Surgical Technology
• Computer-Assisted Surgery

Background:

• Augmented reality (AR) application in surgery is limited by visualization accuracy of internal structures.
• Challenges exist in bowel, liver, gallbladder, and jaw surgeries due to hidden organ visibility.
• Current AR systems require improvements in accuracy and processing speed for clinical use.

Purpose of the Study:

• To enhance cutting accuracy in augmented reality (AR) videos.
• To improve the visualization accuracy of internal structures in surgical AR.
• To reduce the processing time of AR videos for real-time surgical applications.

Main Methods:

• Development of an enhanced block-matching algorithm (BMA).
• Integration of a ghosting map technique with the BMA.
• System designed for improving augmented video processing in surgical contexts.

Main Results:

• Reduced visualization error from 1.67–2.0 mm to 1.48–1.83 mm.
• Improved processing time from 50–58 ms/frame to 59–72 ms/frame.
• Demonstrated significant enhancements in AR video accuracy and speed.

Conclusions:

• The study successfully improved soft tissue reconstruction and visualization in AR videos.
• The enhanced system addresses key limitations for AR in bowel and gallbladder surgeries.
• Findings indicate potential for more accurate and efficient AR-assisted surgical interventions.