Assessing Competence With a Task Trainer: Validity Evidence for Novel Tracheostomy Care Skills Assessment Tool.

Area of Science:

• Medical Simulation
• Healthcare Education
• Respiratory Care

Background:

• Tracheostomy care requires specialized skills for both healthcare providers and caregivers.
• Assessing these essential skills reliably is crucial for patient safety.
• Existing assessment tools may not adequately cover the spectrum of tracheostomy-related competencies.

Purpose of the Study:

• To develop and validate assessment tools for essential tracheostomy skills.
• To evaluate the competency of healthcare providers and caregivers using a simulated task trainer.
• To establish reliable and valid checklists for tracheostomy management skills.

Main Methods:

• Developed three skill checklists: closed suctioning, open suctioning, and tracheostomy change, based on guidelines and literature review.
• Pilot tested and revised checklists through expert review.
• Assessed 64 intensive care staff and 24 caregivers, with interrater reliability testing on a subset.
• Analyzed correlations between performance and staff demographics.
• Conducted a survey to determine automatic fail items and minimum passing scores.

Main Results:

• High interrater reliability was achieved for closed (0.93) and open suctioning (0.93) checklists, and moderate reliability for tracheostomy change (0.76).
• Staff performance correlated with experience for tracheostomy change and inconsistently with discipline.
• A survey of 132 respondents confirmed automatic fail items and an 80% minimum passing score (agreed by 92.9%).

Conclusions:

• Three essential tracheostomy skill checklists were successfully developed.
• These checklists possess multiple sources of validity evidence.
• The tools support simulation-based assessment for ensuring tracheostomy skills competency.