Beyond standard checklist assessment: Question sequence may impact student performance.

Area of Science:

• Medical Education Research
• Clinical Skills Assessment
• Performance Metrics

Background:

• Traditional clinical encounter assessments rely on checklists, often missing nuances of question timing and sequence.
• Direct faculty observation is not always feasible, limiting the capture of dynamic interactional data.
• A novel approach is needed to quantify the sequential aspects of clinical questioning.

Purpose of the Study:

• To determine if question sequence in clinical encounters can be measured using coherence scores.
• To investigate if coherence scores can differentiate between high and low performing candidates.
• To compare coherence scores with existing assessment methods like checklist and global ratings.

Main Methods:

• A standard case checklist for observed structured clinical exams (OSCE) defined a logical question sequence.
• An independent clinician educator provided global ratings for each encounter.
• Coherence scores were calculated based on the observed question sequence and compared to checklist and global ratings.

Main Results:

• Coherence scores demonstrated a positive correlation with both checklist scores and global ratings.
• The correlation between coherence scores and performance metrics strengthened with higher global ratings.
• Coherence scores explained a greater proportion of variance in student performance as global ratings improved.

Conclusions:

• Logically structured question sequences, indicated by coherence scores, may signify higher-performing students.
• Traditional checklist scoring often overlooks the importance of question sequence in evaluating performance.
• The sequence of questions during clinical encounters is a quantifiable performance dimension not captured by current methods.