Assessing the authenticity of the human simulation experience in anesthesiology

There is an absence of systematic study regarding the authenticity of the simulation experience in anesthesia except for bias prone "participant perception" assessments. Forty-two senior level graduate nurse anesthesia students were videotaped as they were exposed to short (20-30 minutes) but intense simulated crisis situations. Three observers with intimate knowledge of the operating room (OR) evaluated the films using a pilot-tested authenticity tool. Agreement among the observers was satisfactory. k statistics ranged from .75-.90 on the various measures (kappa > .70 is considered good). Concerns abounded regarding a failure to convincingly mirror the OR "culture." A high degree of "anticipation" that something was about to go wrong was evident. The sense of being in the "hot seat" was ubiquitous; hypervigilance of monitors exacerbated the unrealism. Rapid, mechanical, and anxious behavior in the anesthesia providers seemed to be excessive. Reviewers rated the case scenarios themselves as extremely realistic and of high educational value. There was concern that the brevity of the scenarios did not reflect the realities of patient care during procedures of average length. Simulation is likely of maximal benefit if the participant perceives it as legitimate and authentic. Our study indicates the authenticity of anesthesia simulation is in evolution with much potential for improvement.