Team training in the skies: does crew resource management (CRM) training work?

The aviation community has invested great amounts of money and effort into crew resource management (CRM) training. Using D. L. Kirkpatrick's (1976) framework for evaluating training, we reviewed 58 published accounts of CRM training to determine its effectiveness within aviation. Results indicated that CRM training generally produced positive reactions, enhanced learning, and promoted desired behavioral changes. However, we cannot ascertain whether CRM has an effect on an organization's bottom line (i.e., safety). We discuss the state of the literature with regard to evaluation of CRM training programs and, as a result, call for the need to conduct systematic, multilevel evaluation efforts that will show the true effectiveness of CRM training. As many evaluations do not collect data across levels (as suggested by D. L. Kirkpatrick, 1976, and by G. M. Alliger, S. I. Tannenbaum, W. Bennett, Jr., & H. Traver, 1997), the impact of CRM cannot be truly determined; thus more and better evaluations are needed and should be demanded.