A Wearable Assistant Device for the Hearing Impaired to Recognize Emergency Vehicle Sirens with Edge Computing

Area of Science:

• Assistive Technology
• Machine Learning for Healthcare
• Wearable Computing

Background:

• People with hearing impairments face significant road safety risks due to their inability to perceive auditory warnings.
• Existing assistive devices may not adequately address the specific dangers posed by traffic and emergency vehicle sounds.

Purpose of the Study:

• To develop a wearable assistant device utilizing edge computing for the hearing impaired.
• To enable recognition of vehicle warning sounds, specifically sirens, to mitigate road hazards.

Main Methods:

• An EfficientNet-based, fuzzy rank-based ensemble model was developed to classify seven audio sounds (vocalizations and sirens).
• The model was embedded in an Arduino Nano 33 BLE Sense for edge computing.
• Audio signals were converted to spectrograms using the short-time Fourier transform for feature extraction.

Main Results:

• The model achieved high accuracy (97.1% offline, 95.2% edge computing) in classifying audio sounds.
• The wearable device successfully detected sirens and alerted users via vibration and OLED display.
• High precision and sensitivity were recorded in both offline and edge computing scenarios.

Conclusions:

• The proposed wearable assistant device demonstrates significant potential in enhancing the safety of individuals with hearing impairments.
• The edge computing implementation offers a practical solution for real-time warning sound detection.
• This technology can help prevent traffic accidents by providing timely alerts to users.