Structure-aware state space modeling with multi-scale feature fusion for railway scene segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Edge-deployed railway safety systems require high-performance semantic segmentation under strict computational and environmental constraints.
• Traditional Convolutional Neural Networks (CNNs) and Transformer models struggle with long-range dependencies and computational complexity for edge applications.
• Existing methods face challenges in processing diverse environmental conditions (fog, rain, nighttime) at high frame rates.

Purpose of the Study:

• To develop a novel deep learning architecture, HybridSeg, for efficient and accurate semantic segmentation in edge-deployed railway safety monitoring.
• To address the limitations of existing models in handling long-range dependencies, computational costs, and varied environmental conditions.
• To integrate reinforcement learning with visual state space models for context-adaptive information propagation.

Main Methods:

• HybridSeg reformulates visual state space modeling as a controllable Markov Decision Process, utilizing reinforcement learning (Proximal Policy Optimization) for meta-learned state space dynamics.
• Incorporates Structure-Aware Deformable Mamba blocks with four-directional scanning and deformable spatial attention for irregular geometry.
• Employs cross-scale attention fusion across four pyramid levels and explicit multi-scale consistency constraints for improved generalization.
• Evaluated on 8,000 railway surveillance images across diverse environmental conditions.

Main Results:

• HybridSeg achieved 92.34% mIoU and 97.82% pixel accuracy on railway surveillance data, outperforming state-of-the-art methods by 1.61-3.16%.
• The model operates at 38.52 FPS with 45.28M parameters, demonstrating 2.31x faster inference than comparable approaches.
• Achieved robust cross-domain transfer with 89.53% CDR and competitive results on Cityscapes (85.73%), CamVid (87.25%), and ADE20K (48.53%).

Conclusions:

• HybridSeg offers a practical and efficient solution for safety-critical edge applications requiring high-accuracy semantic segmentation.
• The novel integration of reinforcement learning and Mamba blocks enables context-adaptive processing and superior performance under extreme constraints.
• The architecture demonstrates significant advancements in real-time processing, accuracy, and generalization for edge AI in railway safety.