Emergency recognition system based on multimodal information

Area of Science:

• Computer Science
• Artificial Intelligence
• Signal Processing

Background:

• Traditional emergency detection methods often rely on single data sources, limiting their accuracy and robustness.
• The increasing need for reliable fall detection and emergency recognition systems in domestic environments.

Purpose of the Study:

• To propose and evaluate a multimodal emergency recognition system integrating image, voice, and gravity sensor data.
• To enhance the accuracy and reliability of emergency situation detection through data fusion.

Main Methods:

• Developing distinct processing modules for image, voice, and gravity sensor data to detect predefined events (e.g., moving, stopping, fainting).
• Implementing a multimodal integration module for fusing event data from individual sensors.
• Incorporating a user re-verification step via voice interaction to confirm emergency situations.

Main Results:

• The proposed system demonstrated robust performance in recognizing emergency situations, specifically faint motions in living room and bathroom settings.
• Experimental results indicate superior performance compared to existing single-modality or less integrated methods.
• The system effectively handles diverse environmental conditions and user states.

Conclusions:

• Multimodal data fusion significantly improves the accuracy and reliability of emergency recognition systems.
• The proposed system offers a robust and effective solution for detecting emergencies in real-world scenarios.
• Future work could involve expanding the range of detectable emergencies and sensor modalities.