Evaluating teaching methods: validation of an evaluation tool for hydrodissection and phacoemulsification portions of
Ronald J Smith1, Colin A McCannel1, Lynn K Gordon1
1From Jules Stein Eye Institute (Gordon, Bartlett, Mondino) and Veterans Administration Medical Center of Greater Los Angeles (Smith, Hollander, Giaconi, Stelzner), Los Angeles, Olive View-UCLA Medical Center (Devgan), Sylmar, and Harbor-UCLA Medical Center (McCannel), Torrance, California, USA.
Purpose:
To develop and assess the validity of an evaluation tool to assess quantitatively the hydrodissection and phacoemulsification portions of cataract surgery performed by residents.
Design:
Case series.
Setting:
Jules Stein Eye Institute, Olive View-UCLA Medical Center, and Veterans Administration Medical Center, Los Angeles, California, USA.
Methods:
The UCLA ophthalmology faculty members were surveyed and the literature was reviewed to develop a grading tool consisting of 15 questions to evaluate surgical technique, including questions from the Global Rating Assessment of Skills in Intraocular Surgery and from the International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubric. Video clips of the hydrodissection and phacoemulsification portions of cataract surgery performed by 1 postgraduate year 2 (PGY2) resident, 1 PGY3 resident, 2 PGY4 residents, and an advanced surgeon were independently graded in a masked fashion by an 8-member faculty panel.
Results:
Eleven of the 15 questions had a significant association with surgical experience level (P<.05, analysis of variance). Interobserver variability in grading yielded intraclass correlation coefficients between 0.28 and 0.72. The questions with the lowest interobserver variability were hydrodissection questions on instrument handling, flow of operation, and nucleus rotation. Nucleus cracking also had low variability. Less directly visible tasks, especially 3-dimensional tasks, had wider interobserver variability.
Conclusions:
Surgical performance can be validly measured using an evaluation tool. Improved videography and studies to identify the best questions for evaluating each step of cataract surgery may help ophthalmic educators more precisely measure training outcomes for improving teaching interventions.
Financial Disclosure:
No author has a financial or proprietary interest in any material or method mentioned.
