Assessing Medical Student Competency in Community-Engaged Service Learning: Development and Validity Evidence of a...

Area of Science:

• Medical Education
• Community Health
• Health Disparities

Background:

• Community-engaged service learning (CSL) is crucial for addressing health disparities and fostering cultural humility in medical students.
• Assessing behavioral and attitudinal skills in CSL remains a challenge, with current methods often limited to reflection and participation.

Purpose of the Study:

• To co-create a multisource feedback tool for assessing observable competencies during the experiential phase of CSL.
• To develop a valid and reliable assessment instrument for medical student CSL experiences.

Main Methods:

• A multisource assessment tool was iteratively developed with community partners and students from 2022-2024.
• Validity evidence was collected using Kane's framework, with pilot testing involving 170 students and 39 site leaders.
• Statistical analyses included exploratory and confirmatory factor analysis, reliability testing (Cronbach's alpha), and correlation analyses.

Main Results:

• The final tool comprised 29 items across 5 domains: cultural humility, communication, teamwork, social responsibility, and professionalism.
• Reliability analyses showed strong internal consistency (α > 0.76 for 3-item constructs) and significant correlations between items (r > 0.47).
• Comparative analysis demonstrated that the tool's subdomain ratings correlated with assessments from other courses with similar objectives.

Conclusions:

• A novel multisource assessment tool was developed and demonstrated preliminary validity for evaluating behavioral and attitudinal skills in CSL.
• This tool and its development process can guide assessment strategies in other CSL programs.
• The findings support the use of multisource feedback for assessing complex competencies in experiential medical education.