Graphical Item Maps: providing clearer feedback on professional exam performance

Area of Science:

• Medical Education
• Assessment Science
• Psychometrics

Background:

• Structured feedback is crucial for learning and assessment, highly valued by candidates.
• Item-specific feedback in high-stakes professional assessments is often infeasible due to cost and item bank stability.
• Graphical Item Maps (GIMs) were explored as a solution for informative feedback in fetal surveillance knowledge assessment.

Purpose of the Study:

• To investigate the feasibility and utility of Graphical Item Maps (GIMs) for candidate feedback.
• To provide informative feedback without compromising the integrity of the assessment item bank.
• To demonstrate the application of GIMs in a high-stakes professional assessment context.

Main Methods:

• Developed Graphical Item Maps (GIMs) for the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP) assessment.
• Utilized Item Response Theory (IRT) models to estimate item and person parameters.
• Mapped the interaction between test-takers and assessment tasks of varying difficulty.

Main Results:

• GIMs are feasible and relatively simple to implement for individual candidate feedback.
• Operational examples from the RANZCOG FSEP assessment demonstrate GIM application.
• The study shows how GIMs can be used by test-takers and educators for assessment feedback.

Conclusions:

• Graphical Item Maps serve as a valuable and insightful feedback tool for clinical practitioners in high-stakes professional education.
• GIMs can inform directed learning by providing clear insights into performance.
• This approach is potentially applicable to similar healthcare professional assessments.