Beyond the Milestones: Data and Decision-Making in Otolaryngology Clinical Competency Committees

The decision-making processes of Clinical Competency Committees (CCCs) in otolaryngology-head and neck surgery (OHNS) are poorly understood. This study explores how CCCs collect, interpret, and use assessment data to evaluate resident performance and progress.

We conducted a cross-sectional qualitative study using focus groups with OHNS faculty who serve on CCCs at the November 2024 Society of University Otolaryngologists meeting. Discussions explored the types of assessment data gathered and how this information is interpreted and applied. Audio recordings were transcribed, de-identified, and analyzed using thematic analysis.

Twenty-one CCC members participated in three seven-person focus groups. Thematic analysis revealed the following findings related to CCC decision-making: (1) There is substantial heterogeneity in the types of data collected and used; (2) Narrative and verbal feedback outweighed structured data; (3) Challenges emerged due to a need to transform informal feedback into documented, defensible data for high-stakes decisions. Gaps in the CCC process included: (1) Collection of data that is ultimately unused in decision-making; (2) A lack of structured mechanisms to encourage faculty engagement and development for performing assessments; and (3) A focus on identifying struggling residents rather than fostering development across all levels of trainee performance.

OHNS CCCs rely on a heterogeneous mix of formal and informal data, with narrative feedback often the most influential in decision-making. The process is challenged by data overload, variable faculty engagement, and a lack of standardization. Addressing such challenges may improve the quality, equity, and transparency of resident assessment and progression decisions.