Related Experiment Videos
Eye metrics as an objective assessment of surgical skill
Lee Richstone1, Michael J Schwartz, Casey Seideman
1The Smith Institute for Urology, The North Shore, Long Island Jewish Health System, New Hyde Park, NY 11042, USA.
Objective:
Currently, surgical skills assessment relies almost exclusively on subjective measures, which are susceptible to multiple biases. We investigate the use of eye metrics as an objective tool for assessment of surgical skill.
Summary Background Data:
Eye tracking has helped elucidate relationships between eye movements, visual attention, and insight, all of which are employed during complex task performance (Kowler and Martins, Science. 1982;215:997-999; Tanenhaus et al, Science. 1995;268:1632-1634; Thomas and Lleras, Psychon Bull Rev. 2007;14:663-668; Thomas and Lleras, Cognition. 2009;111:168-174; Schriver et al, Hum Factors. 2008;50:864-878; Kahneman, Attention and Effort. 1973). Discovery of associations between characteristic eye movements and degree of cognitive effort have also enhanced our appreciation of the learning process.
Methods:
Using linear discriminate analysis (LDA) and nonlinear neural network analyses (NNA) to classify surgeons into expert and nonexpert cohorts, we examine the relationship between complex eye and pupillary movements, collectively referred to as eye metrics, and surgical skill level.
Results:
Twenty-one surgeons participated in the simulated and live surgical environments. In the simulated surgical setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 91.9% and 92.9% accuracy, respectively. In the live operating room setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 81.0% and 90.7% accuracy, respectively.
Conclusions:
We demonstrate, in simulated and live-operating environments, that eye metrics can reliably distinguish nonexpert from expert surgeons. As current medical educators rely on subjective measures of surgical skill, eye metrics may serve as the basis for objective assessment in surgical education and credentialing in the future. Further development of this potential educational tool is warranted to assess its ability to both reliably classify larger groups of surgeons and follow progression of surgical skill during postgraduate training.