A new method for group decision making and its application in medical trainee selection

The problems associated with generating a collaborative ranked preference list represent a common source of dilemma in academic medicine and medical education. Such issues present during the process of choosing among applicants to medical schools, during the selection of postgraduate trainees, and in the course of performance assessments and the prioritising of financial expenditures. Currently, most institutions use pseudo-quantitative methods, such as the averaging of scores awarded on an arbitrary scale. These methods are mathematically problematic and may not accurately reflect group opinion.

The present authors developed a novel algorithm for creating a collaborative preference list that generates and sorts a matrix of pairwise comparisons between applicants or choices without placing any reliance on arbitrary Likert scale-type scores. This method achieves equality in influence across individual assessors, as well as transparency and reproducibility. The authors report a case study of their experience using this new algorithm in the 2013 neonatal-perinatal fellowship match.

When used by this group in the selection of fellowship trainees, the method proposed here allowed for greater efficiency and created a rank-order list that did not require reshuffling or significant debate. A survey of faculty staff and fellows showed much higher levels of satisfaction with the new algorithm and a unanimous desire to use the new algorithm in the future, in preference to a score-based system.

The algorithm developed and described here may reduce arbitrariness in processes that require the collaborative creation of a preference list. This method may have wide applicability in medical education and training, and beyond. The present authors' experience of using this algorithm during the National Resident Matching Program match showed improved perceptions of fairness, ease of use and efficiency.