Competency evaluation using randomized testing: feasibility of a new structured assessment method.

Area of Science:

• Medical Education
• Health Professions Education
• Clinical Skills Assessment

Background:

• Assessing clinical competence in healthcare students is crucial but challenging.
• Direct observation of practical skills is often difficult to standardize and maintain integrity.
• Existing assessment methods may not be learner-centered or competency-oriented.

Purpose of the Study:

• To evaluate the effectiveness of a novel assessment format, Competency Evaluation using Randomized Testing (CERT), for practical skills.
• To determine if CERT is a feasible and valid method for assessing clinical competence in medical students.
• To compare learning outcomes and student satisfaction between CERT and traditional assessment methods.

Main Methods:

• A controlled trial was conducted with final-year medical students (33 in CERT group, 33 in control).
• CERT utilized six stations with four randomized variants each to ensure content integrity.
• Scores were adjusted for variant difficulty, with performance on a standardized posttest as the primary outcome.

Main Results:

• The CERT intervention group showed significantly higher learning scores (mean NBME adjusted scores: 72.9 vs. 61.1, P < 0.001).
• Scores from the CERT format correlated with National Board of Medical Examiners (NBME) marks.
• Medical students rated the new CERT format highly, indicating strong satisfaction.

Conclusions:

• Competency Evaluation using Randomized Testing (CERT) is a feasible and potentially valid format for clinical competence assessment.
• CERT's use of random variations protects assessment integrity, allows flexible scheduling, and facilitates large-scale candidate assessment.
• The study demonstrated improved learning outcomes and high satisfaction with the CERT method in medical education.