Enhancing Situational Awareness with VAS-Compass Net for the Recognition of Directional Vehicle Alert Sounds

Area of Science:

• Engineering
• Computer Science
• Audiology

Background:

• Individuals with hearing impairments face significant risks in traffic due to impaired auditory perception.
• Existing assistive devices like hearing aids and cochlear implants have limitations in cost and functionality.

Purpose of the Study:

• To develop an innovative sound alert assistance system (SAAS) for enhancing situational awareness and road safety.
• To create a cost-effective and accessible solution for individuals with hearing impairments.

Main Methods:

• Proposed the VAS-Compass Net, integrating EfficientNet-lite0, MobileNetV3-Small, and GhostNet convolutional neural networks.
• Utilized a fuzzy ranking ensemble technique for sound classification and direction identification.
• Converted audio signals to spectrogram and Mel-frequency cepstral coefficient images, fused via image stitching.
• Deployed the model on a Raspberry Pi 5 for edge computing, paired with a smartwatch.

Main Results:

• The VAS-Compass Net achieved 84.38% accuracy in server-based computing and 83.01% in edge computing.
• Successfully identified various vehicle alert sounds and their sources' directions.
• Demonstrated feasibility of real-time deployment on a microcomputer for practical application.

Conclusions:

• The developed SAAS effectively enhances situational awareness and alertness for individuals with hearing impairments.
• The system offers a promising approach to significantly improve road safety for this demographic.
• The edge computing deployment enables a practical and accessible solution for daily use.