Assessment consistency in special care dentistry: an exploratory study of structured oral examinations

Assessing competencies in Special Care Dentistry (SCD) is challenging because domain experts are scarce and clinical decision-making is complex. Structured oral examinations (SOEs) may offer a standardized and scalable format for competency-based assessment.

This study aimed to examine whether a SOE supports differentiation between predefined performance levels of SCD-related competencies.

Twenty-four dentists without prior SCD experience independently rated two standardized, video-recorded mock performances representing different competence levels (high vs. moderate) based on the same clinical case. Ratings were performed using a five-point global rating scale (1 = excellent, 5 = fail) supported by a structured expectation guide. Descriptive statistics and inter-rater agreement were examined with particular emphasis on rating distributions and differentiation between performance levels (known-groups approach).

The high-quality performance was rated in the highest performance range by all examiners (23 ratings of 1, 1 rating of 2), whereas the moderate performance received lower ratings (13 ratings of 3; 11 ratings of 2). Rating distributions showed minimal variability and a clear separation between performance levels (median 1 vs. 3). Mean scores were 1.04 (SD 0.20) and 2.54 (SD 0.51), respectively. Although the intraclass correlation coefficient was low (ICC=-0.038), this most likely reflects the highly restricted variance inherent in the study design and does not negate the observed consistency of the rating patterns.

This exploratory study suggests that structured oral examinations may support differentiation between predefined performance levels in SCD under standardized conditions.