Related Experiment Video
Updated: Aug 4, 2025

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
Published on: September 27, 2020
Changing estimates of leadership ability before a programme: retrospective self-assessments and response-shift bias
Oscar Lyons1,2, Harry Kingsley-Smith3, Koot Kotze2
1Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK oscar.lyons@phc.ox.ac.uk.
Background:
Most evaluations of clinical leadership development programmes rely on self-assessments. Self-assessments are vulnerable to response-shift bias. Using retrospective then-tests may help to avoid this bias.In this study, we investigate whether post-programme then-tests (retrospective self-assessments) are more sensitive to change in clinical leadership development programme participants than traditional pre-programme pre-tests when paired with post-test self-assessments.
Methods:
17 healthcare professionals participated in an 8-month single-centre multidisciplinary leadership development programme. Participants completed prospective pre-test, retrospective then-test and traditional post-test self-assessments using the Primary Colours Questionnaire (PCQ) and Medical Leadership Competency Framework Self-Assessment Tool (MLCFQ). Pre-post pairs and then-post pairs were analysed for changes using Wilcoxon signed-rank tests and compared with a parallel multimethod evaluation organised by Kirkpatrick levels.
Results:
A greater number of significant changes were detected using then-test pairs than pre-test pairs for both the PCQ (11 of 12 vs 4 of 12 items) and MLCFQ (7 of 7 vs 3 of 7 domains). The multimethods data showed positive outcomes at all Kirkpatrick levels.
Conclusions:
In ideal circumstances, both pre-test and then-test evaluations should be conducted. We cautiously suggest that if only one post-programme evaluation can be conducted, then-tests may be appropriate means of detecting change.
More Related Videos
08:24The Joint Effect of Social Comparison and Social Distance on Evaluation of Intertemporal Choice Outcomes in Event-related Potential Studies
Published on: August 25, 2023
07:01Working Memory Training for Older Participants: A Control Group Training Regimen and Initial Intellectual Functioning Assessment
Published on: September 20, 2020
Related Concept Videos
Self-Presentation: Self-Monitoring and Self-Handicapping
Hindsight Biases
Stereotype Threat and Self-fulfilling Prophecies
Self-Evaluation: Self-Enhancement and Self-Verification
Fundamental Attribution Error
Self-Report Tests of Personality