Potential applications of latent variable modeling for the psychometrics of medical simulation

Use of simulation-based assessments and training has become increasingly widespread in medicine. It is recognized that simulations can yield a wealth of real-time information about the trainee or examinee's performance, from which inferences about proficiency can potentially be drawn. However, for the inferences to be useful, psychometric evaluation should be conducted and validity evidence amassed. Traditionally, educational and psychological measurement has relied on psychometric models that are static, unidimensional, and based on observed scores. In this article, it is argued that modern psychometric models that are dynamic, multidimensional, and based on latent variables may be useful for evaluating medical simulations. It is also argued that modern computational methods based on Bayesian statistics may provide the technical foundation. Several examples are given and issues for further research are discussed.