A method and resources for assessing the reliability of simulation evaluation instruments
Related Experiment Video
Updated: May 17, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Katie A Adamson1, Suzan Kardong-Edgren
1University of Washington Tacoma Nursing and Healthcare Leadership Program, USA. kadamson@u.washington.edu
Aim:
This article describes a successfully piloted method for facilitating rapid psychometric assessments of three simulation evaluation instruments: the Lasater Clinical judgment Rubric, the Seattle University Evaluation Tool, and the Creighton-Simulation Evaluation Instrument.
Background:
To provide valid and reliable evaluations of student performance in simulation activities, it is important to assess the psychometric properties of evaluation instruments.
Method:
This novel method incorporates the use of a database of validated, video-archived simulations depicting nursing students performing at varying levels of proficiency.A widely dispersed sample of 29 raters viewed and scored multiple scenarios over a six-week period. Analyses are described including inter- and intrarater reliability, internal consistency, and validity assessments.
Results And Conclusion:
Descriptive and inferential statistics supported the validity of the leveled scenarios.The inter- and intrarater reliability and internal consistencies of data from the three tools are provided.The article provides information and resources for readers to access in order to assess their own simulation evaluation instruments using the described methods.
Related Concept Videos
Reliability and Validity
Measures of Intelligence
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
Response Surface Methodology
The process of RSM involves several key steps:
Data Validation
Key parameters for method validation include:
Self-Evaluation Maintenance Model
Quality Assurance