Past-behavioural versus situational questions in a postgraduate admissions multiple mini-interview: a reliability and...

Area of Science:

• Medical Education
• Psychometrics
• Admissions Testing

Background:

• The Multiple Mini-Interview (MMI) primarily utilizes Situational Questions (SQs), unlike traditional Single-Station Personal Interviews (SSPIs) which favor Past-Behavioural Questions (PBQs).
• This study evaluates the reliability and acceptability of incorporating both PBQ and SQ formats within MMI stations for postgraduate medical admissions.

Purpose of the Study:

• To investigate the reliability and acceptability of using both Past-Behavioural Questions (PBQs) and Situational Questions (SQs) within the same station in a postgraduate admissions Multiple Mini-Interview (MMI).
• To compare candidate and examiner perceptions of PBQ and SQ formats in the MMI context.

Main Methods:

• Twenty-six Japanese medical graduates applying to postgraduate specialty training underwent a five-station Accreditation Council for Graduate Medical Education (ACGME)-competency-based MMI.
• Each station included two examiners asking a PBQ followed by an SQ consecutively.
• Reliability analysis was performed on scores from both question types, and participants were surveyed regarding their experience.

Main Results:

• Both PBQ and SQ formats demonstrated high generalizability coefficients (0.822 and 0.821, respectively).
• Seven stations with one examiner per station could achieve a reliability of over 0.80 for both formats.
• Over 60% of candidates and examiners reported positive perceptions of candidate ability, and participants favored the MMI's fairness over SSPIs. Candidates preferred SQs, while examiners found PBQs more relevant.

Conclusions:

• Past-Behavioural Questions (PBQs) and Situational Questions (SQs) are equally reliable and acceptable interview formats for postgraduate admissions MMIs.
• Using both PBQ and SQ formats within the same station with a fixed order may not optimize the MMI's effectiveness as an admissions tool.
• Further research is needed to determine the optimal combination of SQ and PBQ formats to enhance MMI reliability, feasibility, acceptability, and predictive validity.