Assessing the Scholar CanMEDS role in residents using critical appraisal techniques

In this brief report, we describe two ways in which we assessed the Scholar CanMEDS role using a method to measure residents' ability to complete a critical appraisal. These were incorporated into a modified OSCE format where two stations consisted of 1) critically appraising an article and 2) critiquing an abstract.

Residents were invited to participate in the CanMEDS In-Training Exam (CITE) through the Office of Postgraduate Medical Education. Mean scores for the two Scholar stations were calculated using the number of correct responses out of 10. The global score represented the examiner's overall impression of the resident's knowledge and effort. Correlations between scores are also presented between the two Scholar stations and a paired sample t-test comparing the global mean scores of the two stations was also performed.

Sixty-three of the 64 residents registered to complete the CanMEDS In-Training Exam including the two Scholar stations. There were no significant differences between the global scores of the Scholar stations showing that the overall knowledge and effort of the residents was similar across both stations (3.8 vs. 3.5, p = 0.13). The correlation between the total mean scores of both stations (inter-station reliability) was also non-significant (r = 0.05, p = 0.67). No significant differences between senior residents and junior residents were detected or between internal medicine residents and non-internal medicine residents.

Further testing of these stations is needed and other novel ways of assessing the Scholar role competencies should also be investigated.