Speech-Based Activity Recognition for Trauma Resuscitation.

Area of Science:

• Medical Informatics
• Speech Recognition
• Artificial Intelligence

Background:

• Trauma resuscitation involves complex team activities that are challenging to monitor.
• Objective assessment of team performance during resuscitation is crucial for improving patient outcomes.

Purpose of the Study:

• To develop and evaluate a speech-based system for automated recognition of team activities in trauma resuscitation.
• To optimize audio processing and neural network architecture for accurate activity detection.

Main Methods:

• Analysis of audio recordings from trauma resuscitations to characterize dataset properties (activity frequency, noise levels, keywords).
• Evaluation of audio preprocessing techniques, including spectral feature types and audio channels.
• Development of a novel neural network architecture combining a modified VGG network for audio feature extraction and a text-based network for keyword input.

Main Results:

• The proposed system achieved the highest accuracy and average F1-score compared to baseline methods.
• Optimal configuration involved using Mel-spectrum spectral coefficients features with a stereo audio channel.
• Incorporating activity-specific frequent keywords significantly improved recognition performance.

Conclusions:

• Speech-based analysis is a viable method for recognizing team activities in high-stress medical environments like trauma resuscitation.
• The novel hybrid neural network architecture effectively integrates audio and textual information for enhanced activity recognition.
• This technology has the potential to aid in real-time performance monitoring and training for resuscitation teams.