Related Experiment Video
Updated: Jun 29, 2025
Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
Published on: August 9, 2024
A real-time augmented reality system integrated with artificial intelligence for skin tumor surgery: experimental
Kai Huang1,2,3,4,5,6, Jun Liao6, Jishuai He6
1Department of Dermatology.
Background:
Skin tumors affect many people worldwide, and surgery is the first treatment choice. Achieving precise preoperative planning and navigation of intraoperative sampling remains a problem and is excessively reliant on the experience of surgeons, especially for Mohs surgery for malignant tumors.
Materials And Methods:
To achieve precise preoperative planning and navigation of intraoperative sampling, we developed a real-time augmented reality (AR) surgical system integrated with artificial intelligence (AI) to enhance three functions: AI-assisted tumor boundary segmentation, surgical margin design, and navigation in intraoperative tissue sampling. Non-randomized controlled trials were conducted on manikin, tumor-simulated rabbits, and human volunteers in Hunan Engineering Research Center of Skin Health and Disease Laboratory to evaluate the surgical system.
Results:
The results showed that the accuracy of the benign and malignant tumor segmentation was 0.9556 and 0.9548, respectively, and the average AR navigation mapping error was 0.644 mm. The proposed surgical system was applied in 106 skin tumor surgeries, including intraoperative navigation of sampling in 16 Mohs surgery cases. Surgeons who have used this system highly recognize it.
Conclusions:
The surgical system highlighted the potential to achieve accurate treatment of skin tumors and to fill the gap in global research on skin tumor surgery systems.
Insights
This study introduces an AI-powered augmented reality (AR) surgical system for precise skin tumor removal. The system enhances tumor segmentation and surgical navigation, improving treatment accuracy for skin cancer surgery.
Area of Science:
- Medical Technology
- Surgical Innovation
- Artificial Intelligence in Medicine
Background:
- Skin tumors are a global health concern, with surgery being the primary treatment.
- Precise preoperative planning and intraoperative navigation for skin tumor surgery, particularly Mohs surgery, remain challenging and rely heavily on surgeon experience.
Purpose of the Study:
- To develop and evaluate a real-time augmented reality (AR) surgical system integrated with artificial intelligence (AI).
- To enhance AI-assisted tumor boundary segmentation, surgical margin design, and intraoperative tissue sampling navigation for skin tumors.
Main Methods:
- Development of a novel AR surgical system integrating AI for tumor segmentation and navigation.
- Conducting non-randomized controlled trials on manikins, tumor-simulated rabbits, and human volunteers.
- Evaluating system performance in segmentation accuracy, AR navigation mapping error, and clinical application in skin tumor surgeries.
Main Results:
- Achieved high accuracy in benign (0.9556) and malignant (0.9548) tumor segmentation.
- Demonstrated a low average AR navigation mapping error of 0.644 mm.
- Successfully applied the system in 106 skin tumor surgeries, including 16 Mohs surgery cases, with positive surgeon feedback.
Conclusions:
- The developed AR-AI surgical system shows significant potential for accurate skin tumor treatment.
- The system addresses a critical gap in global research for advanced skin tumor surgical systems.
- The technology offers enhanced precision and navigation, potentially improving patient outcomes in skin cancer surgery.
