Learning Analytics of a National Entrustable Professional Activities Platform: Cross-Sectional Study of System-Level...

Competency-based medical education (CBME) relies on entrustable professional activities (EPAs) and Clinical Competency Committee (CCC) deliberation to support defensible decisions about trainee progression. As digital assessment platforms increasingly aggregate workplace-based assessment data across training programs, large-scale learning analytics can provide new insights into how entrustment decisions are generated and interpreted within CBME systems. However, little is known about how national assessment infrastructures influence patterns of entrustment attainment.

This study examined national CCC summative entrustment decisions to identify system-level factors associated with attainment of expected supervision levels across residency training.

We conducted a cross-sectional analysis of nationwide CCC entrustment data derived from the Emyway digital assessment platform used by all accredited otolaryngology-head and neck surgery residency programs in Taiwan. The dataset included 3504 summative entrustment decisions across 12 EPAs for 292 residents. Observed supervision levels were compared with prespecified targets for each training year. Logistic regression models were used to identify factors associated with attainment of expected supervision levels, including training stage, EPA sequencing, CCC review cadence, and program characteristics.

Median supervision levels increased across training years but did not consistently reach expected targets in later stages. Expected entrustment targets were met in 2558 of 3504 decisions (73%). Underattainment was most pronounced in senior training years, particularly R4 and R5. In multivariable analyses, delayed EPA sequencing (adjusted odds ratio 0.75, 95% CI 0.63-0.91) and lower CCC review cadence (adjusted odds ratio 0.41, 95% CI 0.28-0.61) were independently associated with lower attainment. Program-level variability in entrustment outcomes was also observed.

Nationwide learning analytics of CCC entrustment decisions revealed a reproducible late-stage gap between expected and observed autonomy in CBME training. This pattern was associated with features of assessment system design that may be modifiable rather than trainee characteristics alone. National benchmarking of digital assessment data can therefore provide actionable insights to optimize EPA sequencing, CCC governance, and evidence generation within CBME systems.