Legitimation Without Argumentation: An Empirical Discourse Analysis of 'Validity as an Argument' in Assessment

Area of Science:

• Health Professions Education
• Assessment Validity
• Argumentation Theory

Background:

• Validity in health professions education (HPE) assessment is often framed as an argument supporting data interpretation and use.
• Previous scholarship indicates a deficiency in the application of argument and argumentation within HPE.
• This study investigates the discourse surrounding assessment validity and argumentation in published HPE literature.

Purpose of the Study:

• To explore the discourses on argument and argumentation concerning assessment validity within HPE.
• To understand how influential HPE manuscripts shape the conceptualization of validity arguments and argumentation.

Main Methods:

• Employed a bricolage of critical discourse analysis approaches.
• Analyzed a corpus of 39 influential peer-reviewed HPE manuscripts.
• Utilized an analytic framework from Argumentation Theory for coding and theme development.

Main Results:

• The elaboration of argument and argumentation in HPE validity discourse is notably scant.
• Key components of Argumentation Theory, such as intended audience, are largely absent.
• The discourse of 'validity as an argument' is legitimized through authority, institutional action, and narrative, despite limited examination.

Conclusions:

• Corroborates existing findings on the scarcity of argument and argumentation in HPE validity discourse.
• Highlights an opportunity to integrate Argumentation Theory into HPE to enhance validation practices.
• Suggests developing validation practices that actively support the use of argument in HPE assessment.