Do students achieve the desired learning goals using open-book formative assessments?

The present study aimed to examine whether medical students benefit from an open-book online formative assessment as a preparation for a practical course.

A between-subjects experimental design was used: participants - a whole cohort of second-year medical students (N=232) - were randomly assigned to either a formative assessment that covered the topic of a subsequent practical course (treatment condition) or a formative assessment that did not cover the topic of the subsequent course (control condition). Course-script-knowledge, as well as additional in-depth-knowledge, was assessed.

Students in the treatment condition had better course-script knowledge, both at the beginning, t₍₂₁₂₎ = 4.96, p < .01, d = 0.72., and in the end of the practical course , t₍₂₀₈₎ = 4.80, p < .01, d = 0.68. Analyses of covariance show that this effect is stronger for those students who understood the feedback that was presented within the formative assessment, F_{(1, 213)}=10.17, p<.01. Additionally, the gain of in-depth-knowledge was significantly higher for students in the treatment condition compared to students in the control condition, t₍₂₀₈₎ = 3.68., p < .05, d = 0.72 (0.51).

Students benefit from a formative assessment that is related to and takes place before a subsequent practical course. They have a better understanding of the topic and gain more in-depth-knowledge that goes beyond the content of the script. Moreover, the study points out the importance of feedback pages in formative assessments.