Faculty calibration and students' self-assessments using an instructional rubric in preparation for a practical...

The purpose of this study was to investigate the effect of faculty calibration and students' self-assessments on students' performances in a periodontal practical examination.

Before a new instructional rubric was implemented in the second-year periodontics course, faculty calibration was conducted with a pilot group of 32 third-year dental students. Afterwards, the new rubric was implemented in the second-year periodontics course. Second-year students used the rubric for their self-assessments before the practical examination. An intraclass correlation coefficient was used to test the reliability of the faculty members. A paired t test was used to compare scores between self-assessments of the pilot group (third-year students) and faculty evaluation. A two-way analysis of variance was performed to compare scores between self-assessments of second-year students and faculty evaluations. Chi-square tests were used to compare overall failure rates amongst four different classes.

The reliability amongst the faculty members was strong (the ICC = 0.75 at the first and 0.97 at the second calibration). The mean self-assessment score from the pilot group was significantly higher than the faculty evaluation. However, the mean self-assessment score of second-year students was significantly lower than the faculty evaluation. The class that practiced self-assessments with the validated instructional rubric exhibited the lowest overall failure rate compared to three past classes.

Using an instructional rubric and conducting faculty calibration improved the process of the periodontal practical examination. Improving the examination process and practicing self-assessments with feedback from faculty may have a positive impact on students' performances in the examination.