Related Experiment Video
Updated: May 31, 2026
Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
Published on: September 27, 2020
Reflective ability in medical trainees: reliability and practicality of a measurement instrument and assessment in a
Audrey DunnGalvin1,2, Christine L Mai3,4, Jeffrey B Cooper3,4
1School of Applied Psychology, University College Cork, Cork, Ireland.
Purpose:
Characterization and measurement of reflective capacity (RC), the ability to learn from experience to inform future learning and practice, in hospital trainees are critical to the design of medical education programs; testing interventions requires effective assessment. This study assessed RC among anesthesia trainees using the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT), which evaluates reflective writing. The study explored variation by training level, age, program, gender, and examined reliability and validity against an independent reflection measure.
Method:
A prospective observational study (February-May 2021) included anesthesia trainees across 4 training levels, 4 programs, and varied prior RC experience and intended subspecialties. Participants completed the REFLECT instrument (using simulated and personal events) and the Residency Level Cognitive Behavior Survey. Potential influencing factors (scenario realism, perceived importance, prior reflective writing) were examined. Data were analyzed for internal consistency (t-tests, analysis of variance, Pearson's r) and interrater reliability (Cronbach's α, intraclass correlation coefficient).
Results:
Participants (n = 172) demonstrated moderate RC (mean = 2.5/4, standard deviation = 0.45), with no significant differences by training level, program, gender, or age. Greater perceived realism and prior reflective writing were independently associated with higher RC. Those with prior RC training rated reflection as more important and useful. Personal and simulated event scores were similar. REFLECT showed excellent interrater reliability (Cronbach's α = 0.95) and moderate correlation with the Residency Level Cognitive Behavior Survey Reflection domain (r = 0.31, P = .001).
Conclusions:
REFLECT demonstrates robust psychometric properties for assessing RC in anesthesia trainees. Moderate RC scores support the need for interventions to enhance reflection. Although limited to 1 specialty and not assessing application to practice, the instrument's general structure suggests potential utility across postgraduate medical training contexts.
