Assessment of Entrustable Professional Activities Among Dutch Endocrine Supervisors

Area of Science:

• Medical Education
• Endocrinology Training
• Competency-Based Medical Education

Background:

• Entrustable Professional Activities (EPAs) are crucial for individualized medical training in competency-based settings.
• EPAs are increasingly adopted in specialized training, including endocrinology.
• Assessing interrater agreement and factors influencing EPA scores is vital for reliable evaluation.

Purpose of the Study:

• To evaluate the interrater agreement of entrustment decisions for endocrine EPAs.
• To identify factors impacting EPA scoring and decision-making.
• To explore the reliability of EPA assessments in endocrinology residency.

Main Methods:

• A case-vignette study involving standardized written cases across various endocrine topics.
• Nine experienced endocrinologists assessed six case vignettes.
• Analysis included Fleiss Kappa for overall agreement and exploration of five pre-defined factors (capability, integrity, reliability, humility, agency).

Main Results:

• Low interrater agreement was observed: Fleiss Kappa for EPA level was 0.11 and for entrustment decisions was 0.24.
• Significant discrepancies exist between EPA levels assigned by different raters.
• Capability was consistently ranked as the most important factor (56%-67%) influencing entrustment decisions.

Conclusions:

• Current EPA scoring in endocrinology training shows considerable variability among raters.
• Entrustment decisions require multiple observations from a supervisory team.
• Factors beyond direct medical competence, such as capability, significantly influence EPA assessments.