Assessment fairness at the nexus of validity: Linking fairness threats to score meaning

Nearly a century after Kelley's seminal work Interpretation of educational measurements, we revisit the concept of validity to argue that fairness is not a downstream equity add-on but a core requirement throughout the assessment lifecycle. In health professions education, where assessment decisions shape learner progression and patient care, validity arguments must justify the meaning and use of scores for all examinees in equity-relevant contexts.

We advance a fairness-as-validity position: Fairness consists of validity claims about whether scores carry comparable meaning and support defensible decisions across groups. In this framing, fairness threats are validity threats when construct underrepresentation or construct-irrelevant variance systematically alters what scores represent. We distinguish these threats from test misuse and situate fairness evaluation within argument-based validation and the Standards for Educational and Psychological Testing.

Using examples from Objective Structured Clinical Examinations, Multiple Mini Interviews and standardised tests, we show how culturally narrow norms, rater expectations and access barriers introduce avoidable variance that undermines validity. We propose a lifecycle approach that integrates fairness evidence into design, administration, scoring, interpretation and decision-making, drawing on multiple sources of validity evidence and mixed methods. We highlight system levers (e.g. barrier-aware blueprinting, principled accommodations and programmatic assessment) that support defensible interpretation. Finally, we introduce intersectional psychometric approaches to model compounded bias and detect validity threats that are missed when groups are examined in isolation.

Together, these arguments position fairness as central to validity theory and validation practice in socially accountable assessment.