Task model-specific operator skill assessment in routine fetal ultrasound scanning

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Ultrasound Technology

Background:

• Operator skill is crucial for operator-dependent ultrasound scanning, impacting image quality and measurement accuracy.
• Traditional skill assessment methods rely on limited data and may not fully capture competence in complex tasks like biometry.
• The rise of machine learning in healthcare necessitates new ways to define and assess operator skill, especially for image acquisition.

Purpose of the Study:

• To develop and validate a novel neural network-based approach for assessing operator skill in selecting ultrasound frames for biometry.
• To quantify operator competence based on the performance of frame classification models.
• To establish a new definition of skill relevant for computer-assisted ultrasound image acquisition.

Main Methods:

• Development of neural network frame classifiers for specific biometry tasks using a label-efficient training strategy.
• Implementation of a task model-specific network to predict two skill assessment scores based on frame identification probability and classification accuracy.
• Validation using clinically acquired ultrasound data from 139 subjects across two biometry tasks.

Main Results:

• Demonstrated the efficacy of both frame-classification and skill-assessment neural networks.
• The proposed skill assessment method showed a stronger correlation with task model performance compared to existing metrics.
• Results were validated against operator experience metrics, confirming the proposed method's robustness.

Conclusions:

• Task model-specific skill assessment using neural networks is feasible and reliable.
• The proposed objective skill assessment provides a more accurate indicator of task model performance than traditional methods.
• This approach supports the development of AI-driven assistance for ultrasound procedures by objectively evaluating operator skill.