Ensuring Competency of Novice Laparoscopic Surgeons-Exploring Standard Setting Methods and their Consequences

Area of Science:

• Medical Education
• Surgical Simulation
• Laparoscopic Training

Background:

• Proficiency-based simulator training in laparoscopy is advancing.
• However, the impact of various standard-setting methods on trainee assessment remains under-explored.
• Determining adequate proficiency levels for supervised patient surgery is critical.

Purpose of the Study:

• To investigate the consequences of different standard-setting methodologies in laparoscopic simulation.
• To evaluate the proficiency benchmarks required for surgical trainees before patient procedures.

Main Methods:

• Surgical trainees completed the Training and Assessment of Basic Laparoscopic Techniques test.
• Performance was assessed via video recording, focusing on errors and completion time.
• Three standard-setting methods were compared: expert performance, contrasting groups, and modified Angoff.

Main Results:

• Standard-setting methods produced significantly different pass/fail levels (e.g., expert level: 474 points; contrasting groups: 358 points; modified Angoff: 311-386 points).
• Proficiency levels varied, leading to failure rates from 0% to 50% for experienced surgeons and pass rates of 0% to 25% for novices.
• Novice surgeons established higher proficiency standards than experienced surgeons (p=0.008).

Conclusions:

• The choice of standard-setting method critically influences the determination of required laparoscopic proficiency.
• Establishing a consistent and validated standard-setting approach is essential for reliable pass/fail criteria in surgical training.