Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills

Area of Science:

• Surgical Education
• Medical Simulation
• Robotic Surgery

Background:

• Patient safety necessitates validated methods for assessing surgical competency.
• Virtual-reality (VR) simulators offer a promising avenue for training and evaluating robotic surgical skills.
• Objective assessment tools are crucial for ensuring proficiency before patient procedures.

Purpose of the Study:

• To gather validity evidence for a VR simulator test designed for robotic surgical skills.
• To evaluate the simulator's potential as an effective training tool for surgeons.
• To correlate simulator performance with varying levels of robotic surgery experience.

Main Methods:

• Recruited surgeons across novice, intermediate, and experienced robotic surgery levels.
• Utilized the da Vinci Skills Simulator with five standardized exercises.
• Collected performance data over 10 attempts per participant across four sessions.

Main Results:

• A moderate correlation was found between initial simulator scores and prior robotic surgery experience (r=0.58, p=0.0004).
• Significant performance differences were observed between novice and intermediate/experienced groups (p<0.0001).
• All participant groups demonstrated significant performance improvement from the first to the tenth attempt (p<0.00).

Conclusions:

• The VR simulator provides validity evidence for assessing basic robotic surgical skills.
• The simulator is suitable for both competency assessment and as a surgical training tool.
• Further validity evidence is required before utilizing the simulator for high-stakes assessments or certification.