Validity evidence for the communication learning assessment: Prior coursework as a known group

Area of Science:

• Medical Education
• Health Communication

Background:

• Effective physician-patient communication is crucial but difficult to teach and assess.
• The Communication Learning Assessment (CLA) is an AI-assisted tool for evaluating communication skills.
• Validating the CLA is essential to ensure it measures communication competence accurately.

Purpose of the Study:

• To examine the known-groups validity of the CLA.
• To determine if medical students with prior communication coursework exhibit higher patient-centered communication behaviors.

Main Methods:

• 193 medical students completed the CLA, which uses AI to score patient-centered communication behaviors from responses to simulated patient scenarios.
• An independent samples t-test compared behavior counts between students with and without prior communication coursework.

Main Results:

• Students with prior communication coursework (n=103) showed significantly more patient-centered behaviors (M=14.02) than those without (n=90, M=12.84).
• The difference was statistically significant (t(191)=2.81, p=.005, d=0.34).

Conclusions:

• Formal communication training is linked to stronger performance on the CLA.
• CLA scores meaningfully capture communication competence, distinguishing it from general verbal skills.
• The CLA is a promising tool for assessing and improving medical students' communication skills.