Two-stage visual speech recognition for intensive care patients

Area of Science:

• Biomedical Engineering
• Artificial Intelligence
• Clinical Communication

Background:

• Medical procedures like tracheotomy can impair speech while preserving lip movement.
• Effective communication is vital for intensive care patients but often challenging.

Purpose of the Study:

• To develop and evaluate a visual speech recognition framework for silent speech in intensive care units (ICUs).
• To enhance communication between speech-impaired patients and healthcare professionals.

Main Methods:

• A two-stage deep learning architecture was employed for visual speech recognition.
• Facial image frames were used to predict audio features, then text.
• An audio-visual dataset was recorded in an ICU with a clinically relevant corpus.

Main Results:

• The lip-reading system achieved a word error rate of 6.3%.
• This performance level is considered sufficient for practical application.

Conclusions:

• This is the first approach to apply visual speech recognition in an intensive care setting.
• The framework significantly improves communication quality for speech-impaired patients.