Assessing the reliability and validity of a standardized rubric for staff evaluation of all-ceramic crown...

Ensuring a consistent and valid assessment of preclinical technical skills is essential in dental education. This study evaluated the inter-rater reliability and validity of a standardized analytic rubric for staff evaluation of single all-ceramic crown preparations across multiple academic years. It compared student performance in midterm and final examinations, examined cohort variability, and investigated the correlation between academic achievement (grade point average or GPA) and practical performance.

A retrospective cross-sectional study was conducted on fourth-year male dental students from three consecutive academic years (2020-2023). Two experienced double-blinded faculty members independently evaluated midterm and final crown preparations by using a validated analytic rubric. The rubric comprised six criteria: five technical domains (occlusal/incisal reduction, axial reduction, taper, finish line placement, and finishing of the preparation) scored 0-6 points and one professionalism domain scored 0-4 points, yielding a total maximum score of 40 points. Inter-rater reliability, cohort differences, and GPA correlation were assessed using Cronbach's alpha, Kruskal-Wallis tests, and Pearson's correlation, respectively, and significance was set at p < 0.05.

Inter-rater reliability was high for the overall scores (midterm α = 0.881, final α = 0.899). Among the individual parameters, taper (retention and resistance) scored the highest in the midterm examination (Staff 1: 5.13/6), and finish line placement scored the lowest in the final examination (Staff 1: 3.79/6). Professionalism showed the lowest inter-rater consistency (α = -0.095). Performance considerably declined from midterm to final examinations (Staff 1 scores declined from 28.19 to 23.68). Substantial variability existed across the academic years, with the 2021-2022 cohort outperforming the others. GPA showed only one weak positive correlation with final scores from one evaluator (r = 0.287, p = 0.037) and was otherwise not significant.

The analytic rubric proved reliable for standardized preclinical assessment, supporting its implementation despite the unexpected decline in the final examination scores and cohort variations with a high inter-reliability score. Academic GPA was a poor predictor of practical crown preparation performance, reinforcing that cognitive and psychomotor competencies are distinct domains in dental education.