Development of an instrument to assess individual student performance in small group tutorials

Recognizing the need for a valid and reliable method to assess individual tutorial performance in a problem-based learning curriculum, we developed a 31-item instrument from theoretical frameworks and items used elsewhere. A scale was developed for each of three broad learning domains: self-directed learning (SDL), critical thinking (CT), and group process (GP). The instrument demonstrated high internal consistency (SDL = .88, CT = .90, GP = .83) on a sample of 18 tutors and 167 students. Tutor-student interrater reliability coefficients were estimated to be low (SDL = .16, CT = .18, GP = .14) due to lack of variance on the response scale. The instrument showed high correlation (r = .82) with other forms of summative evaluation. In its current form, this standardized and validated instrument is unreliable in differentiating strong from weak tutorial performance but can have a steering effect on student tutorial behaviors. The process of instrument development has general application to other educational programs.