Competency evaluation using randomized testing: feasibility of a new structured assessment method

Assessing the clinical competence of health care students is challenging as it involves direct observation of practical performance. We evaluated the effectiveness of a new format for assessing practical skills: competency evaluation using randomized testing (CERT). CERT is a structured assessment format that uses random variants within each station. CERT is designed to be authentic, learner centered, and competency oriented. We conducted a controlled trial of CERT among final-year medical students in a clinical clerkship. There were 33 medical students in the intervention (CERT) group and 33 in the control group. The CERT sessions included six stations, each using variations to protect the integrity of exam content. Each of the six stations had four variants, which were randomly assigned to students. Scores were adjusted based on the difficulty level of each variant. The primary outcome measure was performance on a standardized posttest. Learning scores were higher in the intervention group [mean National Board of Medical Examiners (NBME) adjusted scores: 72.9 vs. 61.1 in control group; P < 0.001]. Students who entered the session later did not have higher scores compared to those who finished earlier. Scores from the new format correlated with NBME marks. The new format was rated highly by medical students. CERT, a structured assessment using randomized variations in stations, is a feasible and potentially valid format for evaluating clinical competence. By using random variations, CERT protects the integrity of assessment, allows flexible scheduling, and facilitates assessment of a large number of candidates.NEW & NOTEWORTHY Competency evaluation using randomized testing (CERT) is a new format for assessing practical skills. It uses random variations to protect the integrity of the examination content. This allows candidates to be scheduled individually without having to keep them in a waiting room. We evaluated the feasibility of CERT at a medical college and found that it led to improved learning outcomes and high satisfaction ratings. CERT facilitates the assessment of clinical competencies.