Developing a generalizable pediatric ECMO emergency checklist for clinical specialist: Progress and challenges.

Area of Science:

• Critical Care Medicine
• Medical Education
• Cardiovascular Engineering

Background:

• Extracorporeal membrane oxygenation (ECMO) requires specialized clinical specialists (CS) for circuit management during emergencies.
• Existing CS skill assessment tools show significant variation due to differing protocols and circuit designs.
• A need exists for a generalizable ECMO skill assessment with robust validity evidence for summative evaluation.

Purpose of the Study:

• To develop a generalizable ECMO skill assessment tool.
• To enhance the validity and reliability of existing assessment checklists.
• To create a summative evaluation tool for clinical specialist proficiency in ECMO circuit management.

Main Methods:

• A modified Delphi process was used to alter existing checklists and simulations based on a survey of ECMO circuit variations.
• The finalized assessment tool was evaluated for validity and reliability across two institutions and two circuit designs.
• Three raters assessed ten simulations from five subjects, with data analyzed using generalizability (G) and decision (D) studies.

Main Results:

• The generalizability (G) study coefficient was 0, indicating 0% variance attributable to the subject and circuit.
• The largest source of variance was among raters (28.7%).
• Significant variance (27%) was also observed in the relationship between subject and pump type.

Conclusions:

• Despite rigorous development, the ECMO skill assessment demonstrated poor generalizability.
• Lack of familiarity with institution-specific ECMO circuit designs was a major factor limiting generalizability.
• Future efforts should focus on either developing site-specific assessment tools or standardizing ECMO circuit designs for broader applicability.