The argument-based approach to validity: theory, assumptions, evidence, and consequences

Concerns about measurement validation are often expressed as imperatives to use a "valid measure". However, validity is not a characteristic of a measure. Instead, validation and validity refer to score interpretation within a context of use. This is important because responses to health measures can be different in different contexts, influencing equitable consequences of measurement. In this paper, we aim to (1) outline the theory of the argument-based approach to validity, and (2) discuss assumptions and evidence in relation to equitable consequences of measurement.

The argument-based approach to validity asks us to first state how scores will be interpreted and used in context. Assumptions underpinning this statement guide validation planning. Existing and new evidence need to be examined and evaluated in relation to the assumptions and concept of interest, leading to a reasoned evidence-based argument about the degree to which score interpretation in a context of use is valid, with consideration of potential threats to validity and measurement consequences.

Key assumptions are described, including why evidence is needed, what evidence tells us, and the importance of assumptions in relation to the equitable consequences of measurement.

The argument-based approach to validity shifts the focus of validation to a score's interpretation and use in a context, in relation to the concept of interest. A validity argument is built from evidence about the plausibility of the score interpretation in the context of use with consideration of the degree to which measurement consequences will lead to the intended beneficial health consequences and not perpetuate existing inequities in health.