Online examiner calibration across specialties

Integrating undergraduate medical curricula horizontally across clinical medical specialties may be a more patient-centred and learner-centred approach than rotating students through specialty-specific teaching and assessment, but requires some interspecialty calibration of examiner judgements. Our aim was to evaluate the acceptability and feasibility of an online pilot of interdisciplinary examiner calibration. Fair clinical assessment is important to both medical students and clinical teachers METHODS: Clinical teachers were invited to rate video-recorded student objective structured clinical examination (OSCE) performances and join subsequent online discussions using the university's learning management system. Post-project survey free-text and Likert-scale participant responses were analysed to evaluate the acceptability of the pilot and to identify recommendations for improvement.

Although 68 clinicians were recruited to participate, and there were 1599 hits on recordings and discussion threads, only 25 clinical teachers rated at least one student performance, and 18 posted at least one comment. Participants, including rural doctors, appeared to value the opportunity for interdisciplinary rating calibration and discussion. Although the asynchronous online format had advantages, especially for rural doctors, participants reported considerable IT challenges.

Our findings suggest that fair clinical assessment is important to both medical students and clinical teachers. Interspecialty discussions about assessment may have the potential to enrich intraspecialty perspectives, enhance interspecialty engagement and collaboration, and improve the quality of clinical teacher assessment. Better alignment of university and hospital systems, a face to face component and other modifications may have enhanced clinician engagement with this project. Findings suggest that specialty assessment cultures and content expertise may not be barriers to pursuing more integrated approaches to assessment.