A Multimodal In-Ear Audio and Physiological Dataset for Swallowing and Non-Verbal Event Classification.

Area of Science:

• Biomedical Engineering
• Signal Processing
• Machine Learning

Background:

• Swallowing is a key indicator of neurological and emotional well-being.
• Non-invasive, continuous monitoring via ear-worn devices is promising for clinical use.
• Existing audio datasets lack detailed swallowing annotations, hindering research.

Purpose of the Study:

• To create a novel in-ear audio dataset with comprehensive swallowing event labels.
• To facilitate the development of reliable swallowing detection algorithms using ear-worn sensors.
• To demonstrate the feasibility of in-ear audio for swallow classification.

Main Methods:

• Collected synchronized in-/outer-ear audio, ECG, respiration, acceleration, and ultrasound tongue movement data from 34 healthy adults.
• Participants performed diverse verbal and non-verbal tasks in varied acoustic environments.
• Utilized ultrasound signals for accurate swallowing event annotation, validated by experts.

Main Results:

• Developed a new in-ear audio dataset capturing diverse sounds, including detailed swallowing events.
• A fine-tuned neural network using YAMNet embeddings and ZCR features achieved an F1 score of 0.875 ± 0.013 for swallow classification.
• Demonstrated that in-ear audio signals are effective for detecting swallowing events.

Conclusions:

• The novel in-ear audio dataset addresses the critical need for swallowing sound detection resources.
• Ear-worn devices show significant potential for non-invasive swallowing monitoring.
• The study validates the use of in-ear audio and machine learning for accurate swallow classification.