Use of interaction cards to evaluate clinical performance.

Area of Science:

• Medical Education
• Clinical Performance Assessment

Background:

• Current in-training evaluation in Kuwait relies on a global rating scale, which is inconsistent, subjective, and lacks reliability.
• Deficiencies in the global rating scale include positive skewness and poor inter-rater reliability, necessitating a more robust evaluation method.

Purpose of the Study:

• To assess inter-rater variation and reliability of the Interaction Card (IC) method for clinical performance evaluation.
• To measure agreement between the global rating scale and the IC summative evaluation for trainee performance.

Main Methods:

• 370 evaluators assessed 50 trainees across six clinical domains during basic clinical rotations in six hospitals.
• A total of 9146 encounters were analyzed, focusing on clinical skills, professional behavior, case presentation, diagnosis, therapy, and emergency handling.
• Statistical analysis included inter-rater variation assessment, Cronbach's alpha for internal consistency, and Spearman rank correlation/Kappa for agreement.

Main Results:

• The Interaction Card (IC) method showed significant inter-rater variation based on specialty, evaluator rank, and hospital (p < 0.001).
• The IC method demonstrated high reliability with strong internal consistency across the six domains (Cronbach's alpha = 0.914).
• Low correlation (rs = 0.337) and agreement (Kappa = 0.131) were found between the global rating scale and IC summative evaluations.

Conclusions:

• The Interaction Card (IC) method provides reliable and instantaneous formative feedback and summative evaluation for clinical performance.
• The IC method is a valuable tool that can be generalized to various training and examination programs across clinical specialties.
• The findings suggest a shift from subjective global rating scales to more objective methods like the IC for improved medical education assessment.