Related Experiment Video
Updated: Mar 15, 2026
Simulator Training for Endovascular Neurosurgery
Published on: May 6, 2020
Redesign of the System for Evaluation of Teaching Qualities in Anesthesiology Residency Training (SETQ Smart)
Kiki M J M H Lombarts1, Andrew Ferguson, Markus W Hollmann
1From the Professional Performance Research Group, Center for Evidence-Based Education, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands (K.M.J.M.H.L.); Department of Anaesthesia and Intensive Care Medicine, Craigavon Area Hospital, Portadown, United Kingdom (A.F.); Department of Anesthesiology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands (M.W.H.); Centre of Health Sciences Education, Faculty of Health, Aarhus University, Aarhus, Denmark (B.M.); Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles (UCLA), Los Angeles, California (O.A.A.); and UCLA Center for Health Policy Research, Los Angeles, California (O.A.A.). Paracelsus Medical University, Salzburg, Austria European Union of Medical Specialists, Brussels, Switzerland University Medical Center, Utrecht, The Netherlands Diakonessen Hospital, Utrecht, The Netherlands Free University, Amsterdam, The Netherlands Universitätsklinikum Köln, Köln, Germany Universitätsklinikum Köln, Köln, Germany Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands University Hospital of Basel, Basel, Switzerland Erasmus Medical Center, Rotterdam, The Netherlands.
Background:
Given the increasing international recognition of clinical teaching as a competency and regulation of residency training, evaluation of anesthesiology faculty teaching is needed. The System for Evaluating Teaching Qualities (SETQ) Smart questionnaires were developed for assessing teaching performance of faculty in residency training programs in different countries. This study investigated (1) the structure, (2) the psychometric qualities of the new tools, and (3) the number of residents' evaluations needed per anesthesiology faculty to use the instruments reliably.
Methods:
Two SETQ Smart questionnaires-for faculty self-evaluation and for resident evaluation of faculty-were developed. A multicenter survey was conducted among 399 anesthesiology faculty and 430 residents in six countries. Statistical analyses included exploratory factor analysis, reliability analysis using Cronbach α, item-total scale correlations, interscale correlations, comparison of composite scales to global ratings, and generalizability analysis to assess residents' evaluations needed per faculty.
Results:
In total, 240 residents completed 1,622 evaluations of 247 faculty. The SETQ Smart questionnaires revealed six teaching qualities consisting of 25 items. Cronbach α's were very high (greater than 0.95) for the overall SETQ Smart questionnaires and high (greater than 0.80) for the separate teaching qualities. Interscale correlations were all within the acceptable range of moderate correlation. Overall, questionnaire and scale scores correlated moderately to highly with the global ratings. For reliable feedback to individual faculty, three to five resident evaluations are needed.
Conclusions:
The first internationally piloted questionnaires for evaluating individual anesthesiology faculty teaching performance can be reliably, validly, and feasibly used for formative purposes in residency training.
Related Concept Videos
01:20Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
01:15Nursing Evaluation