What works best in a general practice specific OSCE for medical students: Mini-CEX or content-related checklists?

Aim: To develop and pilot a General Practice (GPr) OSCE assessing medical students dealing with patient encounters, which are typical for GPr and to compare different measurement instruments (global ratings, content-specific checklists).Methods: A blueprint based on Entrusted Professional Activities was used to develop prototypical OSCE stations. Four stations were tested with voluntary medical students. Students were videotaped and assessed with self-developed content-specific checklists, a global rating for communication skills, and mini-CEX. Results were compared according to students' phases of studies.Results: All three measurements were able to discriminate between clinical and pre-clinical students. Clearest results were achieved by using mini-CEX. Content-specific checklists were not able to differentiate between those groups for the more difficult stations. Inter-station reliability for the global ratings was sufficient for high-stakes exams. Students enjoyed the OSCE-setting simulating GPr consultation hours. They would prefer feedback from GPs after the OSCE and from simulated patients after each encounter.Discussion and conclusion: Although the OSCE was short, results indicate advantages for using a global rating instead of checklists. Further research should include validating these results with a larger group of students and to find the threshold during the phases of education for switching from checklists to global ratings.