Inferring signs from purposeful samples: The role of context in competency assessment

Area of Science:

• Medical Education
• Psychology
• Assessment Theory

Background:

• Clinical competencies are typically assessed using convenience samples from real-world practice.
• The challenge lies in extrapolating these samples to a student's overall competency profile, considering performance context specificity.

Purpose of the Study:

• To adapt Smith's theory of predictor validity to clinical competency assessment.
• To address the issue of inferring generalizable competencies (signs) from performance samples in medical education.

Main Methods:

• Adapted Smith's theory distinguishing between universals, occupationals, and relationals.
• Applied these concepts to the context of medical student clinical competency assessment.

Main Results:

• Generalizability of clinical competencies from clerkship samples to generic competencies is contingent on their nature: universals, occupationals, or relationals.
• Universals demonstrate the highest generalizability, whereas relationals exhibit the lowest.

Conclusions:

• Medical education needs more nuanced approaches to competency assessment, moving beyond an all-or-none perspective.
• Smith's framework of universals, occupationals, and relationals provides a valuable model for improving assessment, student selection, and career guidance in medicine.