Assessing trainees in the workplace: results of a pilot study

Area of Science:

• Medical Education
• Workplace-Based Assessments
• Physician Training

Background:

• Higher medical training requires robust assessment tools.
• Current assessment methods may not fully capture trainee competency in clinical settings.
• Developing and validating new workplace-based assessments is crucial for effective medical education.

Purpose of the Study:

• To develop and evaluate the utility of novel workplace-based assessments in higher medical training.
• To assess the validity, educational impact, and feasibility of case-based discussion (CbD), acute care assessment tool (ACAT), audit assessment, teaching observation, and patient survey (PS).

Main Methods:

• A prospective pilot study involving UK physician trainees (ST3+) and local study coordinators.
• Data collection through questionnaires and qualitative feedback from trainees and assessors.
• Reliability analysis to determine the required number of assessments for adequate consistency.

Main Results:

• Evidence of validity and positive educational impact was found for all assessed tools (CbD, ACAT, audit assessment, teaching observation, PS).
• Reliability targets were identified: 12 CbDs, 3 ACATs, and 16 PS raters for a coefficient of 0.7.
• Difficulties were encountered regarding the feasibility of implementing the patient survey (PS).

Conclusions:

• The developed workplace-based assessments demonstrate validity and educational value for higher medical training.
• While most tools are feasible, the patient survey requires further refinement to address implementation challenges.
• These findings support the integration of these assessments to enhance physician competency evaluation.