Validity considerations in programmatic assessment of clinical reasoning

Clinical reasoning is a core competency in health professions education, but its assessment is challenging. Clinical reasoning is multifaceted, content- and context-dependent, and largely unobservable, and no single assessment can support a robust characterisation of a learner's clinical reasoning. Programmatic assessment has gained traction as a response to these challenges, but a validity argument explaining why a programmatic approach addresses the specific vulnerabilities of clinical reasoning assessment has received limited attention.

We examine validity issues in the programmatic assessment of clinical reasoning using Kane's argument-based framework, organised around five inferences: performance, scoring, generalisation, extrapolation and consequences. For each inference, we identify the central validity question, describe issues specific to clinical reasoning at the level of the individual assessment and the programme, discuss how a programmatic approach mitigates weaknesses of single assessments and describe residual or new validity threats. A fictional learner followed across undergraduate medical education makes these issues concrete. We then consider generative artificial intelligence (GAI) as a cross-cutting validity challenge affecting each inference.

Programmatic assessment strengthens each inference in the validity chain, but it does not resolve every weakness and introduces new assumptions of its own, particularly around how conflicting evidence is synthesised. GAI compounds these challenges: it can shift what is actually being measured, complicate scoring and generalisation, and blur the line between assessment and target performance, while also offering new opportunities to make clinical reasoning more observable when incorporated deliberately.

We conclude with a three-stage developmental framework-coherence and coverage, intentional depth, and multimodal longitudinal integration-to help institutions integrate existing assessments into a programme of clinical reasoning assessment. This framework reflects validity as an ongoing institutional commitment to evidence building for better learning, decisions and patient care.