Development and Testing of an Instrument for Summative Assessment of Practical Skill Performance: A Generalizability...

Area of Science:

• Nursing Education
• Healthcare Professional Skills Assessment

Background:

• Newly graduated nurses frequently exhibit deficiencies in practical skill performance.
• Current summative assessments for nursing students' practical skills utilize broad, overarching items.
• There is a need for more granular assessment tools to accurately evaluate nursing competency.

Purpose of the Study:

• To develop a detailed instrument for the summative assessment of nursing students' practical skill performance.
• To evaluate the psychometric properties of the newly developed assessment instrument.

Main Methods:

• A comprehensive 50-item instrument was created for skill assessment.
• Video-recorded nursing performances were evaluated by experienced clinical supervisors.
• A multifacet measurement design and generalizability analysis were employed to estimate relevant parameters.

Main Results:

• The study identified that the primary source of measurement error stemmed from the raters, significantly more than from the assessment items.
• Generalizability analysis indicated significant rater variability in skill assessment.

Conclusions:

• Summative assessment of practical nursing skills in realistic settings necessitates the involvement of multiple raters.
• Employing two to three or even four raters is recommended for dependable measurement of most nursing skills.
• Rater training and standardization are critical for improving the reliability of nursing skill assessments.