Looking beyond the individual: Conceptualizing assessment validity as interdependence within teams

Over the last century, validity theory has evolved from Kelley's foundational work on educational measurement through Cronbach and Meehl's construct validation, to Messick's unified framework and, finally, Kane's argument-based approach. Throughout this evolution, one premise remained neglected: The unit of analysis is the individual. Focusing primarily on the individual is problematic, especially in Medicine where constructs like clinical competence are treated as a possession of an individual, and contextual factors such as team configurations are considered statistical nuisances. The clinical landscape has also evolved: Modern patient care is delivered by large, interprofessional teams whose collective competence provides care that no single individual could alone. The disconnect between the primacy of independent performance and the interdependent nature of healthcare delivery poses a validity threat to assessment systems that routinely attribute performance to an individual but fail to account for interdependence, teams and context.

Growing evidence suggests that the interdependence of clinical work is not noise obscuring one's 'true' competence, but rather a fundamental part of how competence emerges and shifts over time. If competence is relationally situated, then identifying who is on a team becomes a foundational inference. Our work developing team identification approaches in inpatient settings demonstrates the difficulty of this inference, as there is often no granular and accurate record of patients' care teams available. Electronic health records and their associated metadata may provide a way forward.

The next century of validity theory must grapple with inferences about individuals embedded within teams. Medical education needs to understand that looking beyond the individual is necessary to ensure valid interpretations of one's competence. Starting with generalizability studies that incorporate team composition as a measurement facet, expanding conceptual work about relational competence, and investigating the extent to which team identification can directly measure relational competence, all position medical education to accurately measure individual competence by considering the teams that surround and contribute to the performance of individuals.