Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

The relationship between interviewers' characteristics and ratings assigned during a multiple mini-interview.

Kevin W Eva1, Harold I Reiter, Jack Rosenfeld

  • 1Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada. evakw@mcmaster.ca

Academic Medicine : Journal of the Association of American Medical Colleges
|May 29, 2004
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

With great power comes great responsibility: How narrow conceptions of validity in high-stakes testing undermine competence.

Medical education·2026
Same author

How Did You Get There? The Value of Segues for Illustrating Your Logic.

Perspectives on medical education·2026
Same author

Consistency and inconsistency with which sociodemographic variables are associated with performance on medical school selection tools.

Medical teacher·2026
Same author

It is the Unknown That Matters: Program Directors' Perspectives on Information Gaps in Learner Educational Handovers.

Academic medicine : journal of the Association of American Medical Colleges·2026
Same author

The impact of GenAI on applicant behaviour, performance, and interview reliability during virtual interviews for medical school admissions.

NPJ digital medicine·2025
Same author

How can I help at this moment? Outlining three generations of coaching for health professions educators.

Medical education·2025

The Multiple Mini-Interview (MMI) shows good reliability for assessing personal qualities in medical school admissions. Increasing interview exposure is more effective than adding interviewers for consistent candidate evaluation.

Area of Science:

  • Medical Education
  • Admissions Assessment
  • Psychometrics

Background:

  • Traditional interviews face challenges in consistent candidate evaluation.
  • The Multiple Mini-Interview (MMI) protocol was developed to enhance the objectivity of admissions processes.
  • Assessing the reliability of diverse rater groups within the MMI is crucial for its validity.

Purpose of the Study:

  • To evaluate the consistency of ratings between health sciences faculty and community members during the MMI.
  • To determine the impact of rater composition (faculty, community, or mixed) on rating reliability.
  • To compare the divergence of ratings across different rater pairings within the MMI framework.

Main Methods:

  • A nine-station Multiple Mini-Interview (MMI) was administered to 54 undergraduate MD program candidates.

Related Experiment Videos

  • Raters included pairs of faculty members, pairs of community members, and mixed pairs.
  • Generalizability Theory was applied to analyze rating consistency across different rater subgroups.

    • Main Results:

    • The overall test reliability for the MMI was calculated at .78.
    • A Decision Study indicated that increasing the number of interviews per candidate is more resource-efficient than increasing interviewers per interview.
    • Rating divergence was highest between community and faculty members and lowest among community members. Participants reported positive experiences with the MMI.

    Conclusions:

    • The MMI serves as a reliable protocol for assessing candidate personal qualities, acknowledging context specificity through multiple sampling.
    • Enhancing interviewer diversity in the MMI may lead to a more heterogeneous cohort of accepted candidates.
    • Further research is needed to ascertain the validity of judgments from different rater groups.