ERCP assessment tool: evidence of validity and competency development during training

Area of Science:

• Gastroenterology
• Medical Education
• Surgical Skills Assessment

Background:

• The endoscopic retrograde cholangiopancreatography (ERCP) direct observation of procedural skills (DOPS) is a 27-item tool designed to support ERCP training in the UK.
• This study aimed to evaluate the validity and reliability of the ERCP DOPS tool and assess competency development during training.

Purpose of the Study:

• To validate the ERCP DOPS tool for assessing competency in ERCP procedures.
• To evaluate the learning curve and competency development of trainees using the ERCP DOPS tool.
• To identify predictors of DOPS competence in ERCP trainees.

Main Methods:

• A prospective study analyzed 818 ERCP DOPS from 109 trainees across 80 UK centers between July 2016 and October 2018.
• Reliability was assessed using Cronbach's alpha, and competency benchmarks were established using the contrasting groups method.
• DOPS scores were stratified by lifetime procedure count to evaluate learning curves, and multivariable analyses identified predictors of competence.

Main Results:

• The ERCP DOPS demonstrated high reliability (Cronbach's alpha = 0.961).
• An optimal competency benchmark was achieved at 87% of assessed DOPS items, with overall competence reached after 200-249 procedures.
• While selective cannulation competence was achieved by 300 procedures, complex therapeutic interventions like stenting, sphincterotomy, and sphincteroplasty required more experience.

Conclusions:

• The ERCP DOPS tool is a valid and reliable measure for assessing procedural competency in ERCP training.
• Trainees require a minimum of 300 hands-on ERCP procedures to achieve competence for summative assessment and independent practice.
• Learning curves vary for different ERCP components, highlighting the need for sufficient procedural volume before independent practice.