Past-behavioural question-based multiple mini-interviews: A competency-aligned, predictive validity study

Multiple Mini-Interview (MMI) predictive validity studies rarely compare past-behavioural question (PBQ) and situational question (SQ) formats or use competency-aligned workplace-based assessments. This study evaluated which format yielded better predictive validity, and whether a resource-efficient MMI using the superior format demonstrates sustained predictive validity against competency-matched multisource feedback (MSF).

For Japanese junior trainee recruitment, a rolling, three-station MMI assessed three non-cognitive competencies: patient safety and quality assurance (PSQA), interpersonal network building (INB), and professionalism (Pro). Phase 1 (n = 26) compared PBQ versus SQ predictive validities. Based on Phase 1 results, Phase 2 (n = 65) used the PBQ-only format. Pearson's correlations, deattenuated and corrected for range-restriction, examined relationships between MMI and MSF across overall, and non-cognitive competencies, at year-one and year-two. Furthermore, an exploratory analysis examined correlation between MMI and the cognitive MSF competency, problem solving (PS).

Phase 1 established PBQ superiority (corrected r, r_c = 0.44, p = 0.001) over SQ. In Phase 2, the overall PBQ-based MMI significantly predicted overall year-one MSF (r_c = 0.52). For the aligned framework, competency-specific correlations were highly significant for PSQA (r_c = 0.81) and INB (r_c = 0.60). All the above predictive validities persisted or increased in year-two, whereas Pro demonstrated weak, non-significant correlations at both year-one (r_c = 0.17), and year-two (r_c = 0.19). Correlation between overall MMI and MSF PS competency revealed favourable results (r_c = 0.51 at year-one, persisting in year-two).

PBQ emerged as the better MMI format over SQ. A resource-efficient, PBQ-based postgraduate MMI produced robust, sustained predictive validity over two years when evaluated against a competency-aligned MSF criterion, alongside an exploratory MSF PS competency. The lack of correlation in the Pro competency underscores the inherent difficulty of baseline assessment, suggesting the need for more professionalism stations.