7 Fellows, 7 Fellowships: Why N = 1 Matters in Program Evaluation

Program evaluation data collected from learners are widely used in medical education for quality improvement, accountability, and accreditation. Generally, success entails high response rates and mean item scores above predetermined thresholds. Although pragmatic, these practices assume when data from multiple learners are aggregated, they converge on a more accurate representation of a single educational object, such as a training program, and reduce the influence of misrepresentative, outlying data. Drawing on Communities of Practice and the professional identity formation literature, this Perspective argues learner-completed program evaluations are better understood as traces of learner-program interactions, shaped by each learner's experience of socializing into a community of practice. Because trainees enter programs with pre-existing personal and professional identities, participation in training is never fully identical from one learner to the next. Rather than capturing multiple data points about a "single" program, evaluation responses describe multiple fellowships, each of which emerged through the interaction between a particular learner and the program's design. This reconceptualization has important implications for the interpretation of isolated ("N of 1") findings. Outliers, especially when they suggest a negative experience, should not be dismissed merely as a minority view, as they capture meaningful experiences obscured by aggregate reporting. This requires reconsideration of how educational data are interpreted, how evaluation systems preserve variation, and how curricula are designed and delivered.