Virtual reality and written assessments are of potential value to determine knowledge and skill in flexible...

Technically challenging professions such as those of the defense and transportation industries increasingly use computer-based simulation and written self-learning instruments for education and to determine competency. A structured learning curriculum does not exist, however, for flexible bronchoscopy, a minimally invasive diagnostic procedure performed on thousands of patients by respiratory specialists, otolaryngologists, anesthesiologists, and surgeons worldwide.

To explore an analogous strategy of measuring theoretical knowledge in flexible bronchoscopy and specific technical skills using written knowledge assessments and a virtual reality bronchoscopy simulator.

Twelve trainees from a university pulmonary medicine training program were asked to identify and enter five specific bronchial segments on command using a virtual reality bronchoscopy skill station, and to complete a 50-question examination pertaining to bronchoscopy theory. Their performance scores and opinions pertaining to the use of these methods of assessment were then recorded.

Trainees correctly identified and entered 71% of the bronchial segments required on command (median 60%, range 40-100%). Fifty percent (3/6) of the trainees who had performed more than 200 flexible bronchoscopies successfully entered all segments required. None (0/6) of those who had performed less than 200 flexible bronchoscopies correctly located, identified and entered all required segments. Despite disparate performance, all trainees believed that technical skills could be improved through practice and instruction using computer-based simulation. On the written assessment, only 51% of questions were answered correctly (median 52%, range 32-60%). No relationship between technical skill and theoretical knowledge was noted. In addition, neither bronchoscopy skill nor theoretical knowledge were associated with years of training or number of bronchoscopies previously performed.

Trainees concluded that (1) bronchoscopy simulation was realistic, (2) simulator-based practice would help improve technical skills, and (3) a written questionnaire would benefit theoretical knowledge acquisition if designed as a learning instrument. The wide variability noted in this study as well as the lack of a relationship between technical skill, knowledge of bronchoscopy theory, extent of training, and bronchoscopy experience suggest that competency should not be assumed based on years of bronchoscopy training or on an arbitrary number of procedures performed.