Real-Time Acoustic Detection of Critical Incidents in Smart Cities Using Artificial Intelligence and Edge Networks

Area of Science:

• Computer Science
• Signal Processing
• Artificial Intelligence

Background:

• Real-time audio surveillance requires integrating diverse technologies for effective sound event detection and analysis.
• Existing systems often face challenges in accuracy, data processing, and complex query capabilities.

Purpose of the Study:

• To develop a real-time, distributed audio surveillance system.
• To enable accurate acoustic classification and analysis of soundscapes using advanced AI models.
• To facilitate complex querying of audio events and their relationships within an environment.

Main Methods:

• A network of ESP32 microcontrollers with microphones captures and transmits compressed audio data via MQTT.
• Raspberry Pi 5 devices run an audio transformer model (trained on AudioSet) for real-time event classification and timestamping.
• Detected events are stored in a database, converted to JSON, and parsed into a graph structure for analysis with Python and large language models (LLMs).

Main Results:

• The system achieves high accuracy in real-time audio event classification.
• A novel graph parsing method demonstrated low false-alarm rates.
• In movie audio analysis, the system achieved 0.882 accuracy, 0.8 precision, 1.0 recall, and 0.89 F1 score.

Conclusions:

• The integrated system offers robust distributed sensing and precise transformer-based audio classification.
• Treating audio as text enables advanced applications in acoustic surveillance and environmental monitoring.
• The approach provides a foundation for sophisticated analysis of dynamic audio environments.