Locally adapting generic rubrics for the implementation of outcome-based medical education: a mixed-methods approach

Rubrics are frequently used to assess competencies in outcome-based medical education (OBE). The implementation of assessment systems using rubrics is usually realised through years of involvement in projects with various stakeholders. However, for countries or specialities new to OBE, faster and more simplified processes are required. In March 2019, Japan introduced nine competencies and generic rubrics of competencies for medical residents. We explored the local adaptation of these generic rubrics and its consequences for assessors.

The study followed three steps. First, we locally adapted the generic rubrics. This was followed by conducting mixed-method research to explore the effect of the local adaptation. In step two, we examined the correlations between the scores in the locally adapted assessment sheets for supervising doctors and generic rubrics. In step three, we conducted interviews with supervising doctors. The study was conducted in the General Internal Medicine Department of Nagoya University, Japan. In the first step, doctors in the Medical Education Center and other medical departments, clerks, and residents participated. Supervising doctors in the General Internal Medicine Department participated in the second and third steps.

A locally adapted assessment system was developed and implemented in seven months. The scores of the generic rubrics and the adapted assessment tool completed by the supervising doctors showed good correlations in some items as opposed to others, assessed mainly with other tools. Participant interviews revealed that local adaptation decreased their cognitive load leading to consistent ratings, increased writing of comments, and promoting reflection on instruction.

This adaptation process is a feasible way to begin the implementation of OBE. Local adaptation has advantages over direct use of generic rubrics.