Evaluating intraoperative laparoscopic skill: direct observation versus blinded videotaped performances

Area of Science:

• Surgical Education
• Medical Simulation
• Surgical Skill Assessment

Background:

• Direct observation tools like the Global Operative Assessment of Laparoscopic Skill (GOALS) are established for evaluating surgical proficiency.
• Assessing laparoscopic skills through recorded performances offers a flexible alternative to direct observation.

Purpose of the Study:

• To determine the reliability and validity of the GOALS instrument when applied to blinded, videotaped surgical procedures.
• To evaluate the impact of rater experience and training on the assessment of laparoscopic skills using GOALS.

Main Methods:

• Five novice and five experienced surgeons' laparoscopic cholecystectomies were videotaped.
• Four laparoscopists evaluated the videos using GOALS, with two having prior GOALS experience.
• Interrater reliability (intraclass correlation coefficient) and construct validity (2-way ANOVA) were analyzed.

Main Results:

• Interrater reliability between video raters and direct raters was 0.72.
• Intraclass correlation among video raters was 0.68.
• Most raters could distinguish between novice and experienced surgeons, indicating construct validity.

Conclusions:

• The GOALS instrument demonstrates reliability and validity for assessing laparoscopic skills from videotaped performances.
• Rater training and prior experience with GOALS appear critical for consistent and accurate skill evaluation.
• Video-based assessment using GOALS is a viable method for surgical skill evaluation, with potential benefits for training evaluators.