Triple jump examinations for dental student assessment

The triple jump examination (TJE) attempts to assess a higher level of learning with demand for analysis, critical thinking, and resolution of problems presented by written scenarios based on patient care situations. The purpose of this study was to examine the internal consistency, scale reliability, and interrater reliability of the TJE used at the Ostrow School of Dentistry, University of Southern California. On the sample of 2,227 examinations administered by seventy-seven raters across a three-year time period, the Cronbach's coefficient alpha for internal consistency of the overall TJE was found to be good (a=0.869). The internal consistency of the three subscales was found to be acceptable (a=0.731), good (a=0.820), and good (a=0.820). Average and single measures intraclass correlation coefficients (ICC) for scale reliability were significant at p<0.001, indicating strong interrater reliability. There were no statistically significant differences (p≤0.05) in the mean scores assigned on the TJE between rater groups defined by rater experience level with the TJE. A very high level of agreement among rater pairs was also observed. Across the entire three-year study period, with over 19,152 ratings, the seventy-seven raters were in general agreement 99.5 percent of the time and in exact agreement 77.2 percent of the time.