Evaluating Student Clerkship Performance Using Multiple Assessment Components

Area of Science:

• Medical Education
• Assessment in Medical Training

Background:

• Family medicine clerkships require diverse assessment tools to evaluate a broad range of objectives.
• A single assessment cannot capture the full scope of student learning.
• Ensuring the longitudinal accuracy and validity of assessments is crucial.

Purpose of the Study:

• To investigate the relevance and validity of various assessment tools used in family medicine clerkships.
• To understand the correlation between different assessment methods in evaluating medical students.
• To assess the psychometric properties of family medicine clerkship assessments.

Main Methods:

• Retrospective correlation analysis of student scores across multiple assessment tools.
• Comparison of individual assessment scores for 267 family medicine clerkship students.
• Defined correlation thresholds: weak (ρ<0.3), moderate (0.3-0.5), high (ρ>0.5).

Main Results:

• Low correlations were observed between clinical evaluations and case-based short-answer questions (ρ=0.165) and objective structured clinical examinations (OSCEs) (ρ=-0.246).
• Unexpectedly high correlation was found between multiple-choice questions and OSCEs.
• Preliminary analysis with a standardized exam showed weaker correlation with OSCEs, as expected.

Conclusions:

• Low correlations between assessments are expected as they measure different objectives, supporting discriminant validity.
• Higher correlations between component scores suggest convergent validity.
• Replacing in-house multiple-choice questions with a standardized exam improved assessment validity, highlighting the value of periodic review and standardized testing.