A structured deliberation framework for Clinical Competency Committees: A simulation-based study

Clinical Competency Committees (CCCs) make high-stakes summative entrustment decisions; however, variability persists when deliberative standards remain implicit and inconsistently applied. We conceptualized this divergence as arising from unstructured evidence-to-judgment processes and misaligned evaluative expectations. This study examined the use of a structured deliberation scaffold to support CCC decision-making in a simulation-based faculty development setting.

We conducted a theory-informed, simulation-based educational intervention during a national faculty development workshop. Seventy-two faculty members from 32 otolaryngology residency programs reviewed simulated CCC cases and evaluated deliberation quality using the 12-item Objective Structured Clinical Competence Committee Evaluation (OSCCCE) checklist. One scripted video case (Resident B) was designed a priori to represent principle-concordant deliberation. Participants then worked in seven small-group role-play simulations to make summative entrustment decisions across 12 EPAs. Differences in OSCCCE scores and inter-group agreement in entrustment decisions were analyzed.

Structured process quality differed significantly across simulated cases. The structured, principle-concordant video case (Resident B) and the actively facilitated role-play simulation (Resident C) received significantly higher OSCCCE scores than the unstructured video case (Resident A) across all domains. Participants also rated collective evaluation processes as more useful in supporting resident development and curriculum improvement. Inter-group agreement in entrustment decisions across the seven simulated CCC groups was substantial for the standardized simulated case (Gwet's AC2 = 0.707, 95% CI 0.566-0.848; p < .001).

A structured simulation-based approach may help make CCC deliberative processes more explicit and observable in faculty development settings. These findings suggest that process-focused training may support transparency and shared understanding in programmatic assessment, but should be interpreted as preliminary simulation-based evidence rather than proof of improved real-world reliability or decision accuracy. Future work should examine transfer to live CCC settings and longer-term impact on decision stability and feedback quality.