Triple jump examination evaluation of faculty examiners by dental student examinees

The triple jump examination (TJE) is an oral examination that poses challenges for objective assessment. Student satisfaction levels with faculty assessment can provide information on quality of teaching and students' perceptions of the learning environment. The purpose of this study was to determine scale and interrater reliability of an instrument used by approximately 576 first-year dental students at one U.S. dental school for assessment of their faculty evaluators following midterm and final TJEs over a three-year period. One hundred and one faculty members served as administrators of the TJE with a range of one to 187 times (mean=44.10, median=29, mode=11). The grand mean for six items on a six-point Likert scale was 5.39 with a pooled standard deviation of 1.01. Results indicate positive agreement toward performance of examiners with strong interrater reliability (Average Measures ICC=0.936, Single Measures ICC=0.708) (F5,23475 = 51.564, p<0.001) and consistency across all items (Cronbach's α=0.936). The a priori assumption that students would rate faculty higher as they gained experience over time was rejected (r=0.018, p=0.429). Indications are that faculty members, who are relatively inexperienced with TJE examining, can perform acceptable assessment from the students' perspective. Overall, these students expressed a high level of satisfaction with TJE faculty performance.