Vision-based augmented reality guidance for setting up robot-assisted spine surgery.

Area of Science:

• Medical Robotics
• Computer-Assisted Surgery
• Augmented Reality

Background:

• Robot-assisted surgery enhances precision but faces setup challenges.
• Augmented reality (AR) offers visual guidance but often requires complex calibration.
• Current AR systems necessitate additional steps for head-mounted display and imaging system calibration.

Purpose of the Study:

• To develop an AR guidance approach for robot-assisted procedures using only RGB video for spatial calibration.
• To minimize setup effort and complexity in AR-guided robotic surgery.
• To enable markerless AR guidance through efficient spatial calibration.

Main Methods:

• Established a shared coordinate frame between AR headset and intraoperative imaging system.
• Utilized a Vision Foundation Model (VFM) to estimate inter-device poses directly from RGB video.
• Applied multi-frame Umeyama optimization with RANSAC for refining scale, rotation, and translation in challenging scenarios.

Main Results:

• Achieved sub-3° rotation and 15 mm mean translation error compared to ArUco-based methods.
• Demonstrated markerless calibration, eliminating the need for fiducial markers.
• Verified accurate needle placement within the robot's workspace, allowing for fine adjustments.

Conclusions:

• Vision-based calibration enables accurate, markerless AR guidance for robotic procedures.
• The proposed approach significantly reduces setup complexity.
• This method shows feasibility for seamless AR integration in robotic surgery.