Observed consultation: confidence and accuracy of assessors

Area of Science:

• Medical Education
• Psychology of Assessment

Background:

• Assessor judgments are crucial for evaluating medical students.
• Understanding the relationship between assessor accuracy and confidence is vital for reliable assessments.

Purpose of the Study:

• To investigate the accuracy of assessor judgments in medical student consultations.
• To examine the level of assessor confidence in their judgments.
• To explore the correlation between assessor accuracy and confidence.

Main Methods:

• Assessors viewed recorded medical student consultations.
• Scoring included a checklist, consultation attributes, and a passmark scale with confidence ratings.
• Stimulated recall interviews were used to understand scoring rationale.

Main Results:

• Assessor accuracy and confidence were lowest for student performances near the passmark.
• Over-confidence was highest for borderline performances; under-confidence was observed at performance extremes.
• Factors influencing scoring and confidence were identified through interviews.

Conclusions:

• Assessor confidence and accuracy are not always aligned, particularly for borderline cases.
• Over-confidence increases with judgment difficulty, but under-confidence can occur for less difficult judgments.
• Perceived need for more information may boost confidence without improving accuracy, potentially increasing over-confidence.