Assessing competency in less invasive surfactant administration: simulation-based validity evidence for the LISA-AT...

Area of Science:

• Medical Simulation
• Respiratory Care
• Neonatal Medicine

Background:

• The Less Invasive Surfactant Administration Assessment Tool (LISA-AT) was developed to standardize operator training and competence assessment.
• Validity evidence is crucial to support the use of LISA-AT scores in educational and assessment contexts.

Purpose of the Study:

• To gather validity evidence for the LISA-AT within a simulated environment.
• To establish the reliability and discriminatory power of the LISA-AT metrics.

Main Methods:

• Validity evidence was collected using the Messick framework.
• The minimum passing score was determined by the lowest quartile of expert LISA-AT scores.
• Ten experts and 23 novices participated in the assessment.

Main Results:

• Eight of 15 LISA-AT metrics effectively differentiated between novices and experts.
• The tool demonstrated high test-retest reliability (Spearman's rho = 0.87) and internal consistency (Cronbach's alpha = 0.88).
• Novices achieved a median score of 39/40 after six attempts, but their laryngoscopy skills and time differed significantly from experts.

Conclusions:

• The LISA-AT provides robust validity evidence for training and assessing LISA operator competence in simulated settings.
• A minimum passing score of 39/40 ensures a baseline level of competence.
• While the LISA-AT effectively trains novices, further focus on advanced skills like laryngoscopy may be necessary before clinical application.