A self-attention driven multi-scale object detection framework for adverse weather in smart cities.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Smart City Technology

Background:

• Object detection systems struggle with adverse weather (fog, rain, snow), leading to performance degradation.
• Reduced visibility and background noise obscure object boundaries, impacting road safety and urban security.

Purpose of the Study:

• To propose a Self-Attention Driven Multi-Scale Object Detection Framework for reliable object detection under adverse weather conditions.
• To enhance the performance of existing object detection models in challenging environmental scenarios.

Main Methods:

• Integrated pipeline combining Adaptive Dual-Background Model (ADBM), Pixel-based Finite State Machine (PFSM), and Attention-based Scale Module (ASM) with Self-Attention Optimized You Only Look Once (SAO-YOLO).
• Holistic interaction among background refinement, feature selection, and attention-guided detection for stable, context-aware predictions.
• Thermal-visual feature fusion at the P3 level of Feature Pyramid Network (FPN) for improved visibility robustness.

Main Results:

• Achieved 97.25% Accuracy, 95.33% Precision, 96.37% Recall, and 96.55% F1-score on the Detection in Adverse Weather Nature dataset.
• Outperformed state-of-the-art models including YOLOv5, YOLOv8, and DEtection TRansformer.
• Demonstrated superior detection reliability under complex weather variations.

Conclusions:

• The integrated ADBM-PFSM-ASM-SAO-YOLO framework effectively maintains high detection reliability in adverse weather.
• The proposed model offers a practical solution for real-time traffic monitoring, urban surveillance, and public safety applications.
• The study highlights the importance of integrated approaches for robust object detection in uncertain environments.