Evaluating the effectiveness of training strategies: performance goals and testing

The Public Health Service policy, Animal Welfare Act regulations, and the Guide for the Care and Use of Laboratory Animals all require that institutions provide training for personnel engaged in animal research. Most research facilities have developed training programs to meet these requirements but may not have developed ways of assessing the effectiveness of these programs. Omission of this critical activity often leads to training that is ineffective, inefficient, or unnecessary. Evaluating the effectiveness of biomedical research and animal care training should involve a combination of assessments of performance, competence and knowledge, and appropriate tests for each type of knowledge, used at appropriate time intervals. In this article, the hierarchical relationship between performance, competence, and knowledge is described. The discussion of cognitive and psychomotor knowledge includes the important distinction between declarative and procedural knowledge. Measurement of performance is described and can include a variety of indirect and direct measurement techniques. Each measurement option has its own profile of strengths and weaknesses in terms of measurement validity, reliability, and costs of development and delivery. It is important to understand the tradeoffs associated with each measurement option, and to make appropriate choices of measurement strategy based on these tradeoffs arrayed against considerations of frequency, criticality, difficulty of learning, logistics, and budget. The article concludes with an example of how these measurement strategies can be combined into a cost-effective assessment plan for a biomedical research facility.