Understanding and predicting rolling-gap pedestrian behavior under mixed traffic using pose-informed deep learning

Pedestrian safety remains a critical concern in developing countries, particularly at unsignalized intersections characterized by non-lane-based, heterogeneous traffic. A prevalent and risky behavior in such contexts is rolling-gap crossing, where pedestrians opportunistically navigate through narrow moving-traffic gaps, often pausing mid-crossing. This study aims to develop a pose-based deep learning framework to predict, in real time, whether a pedestrian will wait or continue walking during rolling-gap crossings.

Video data from two urban intersections in India were analyzed using an automated pipeline integrating YOLOv11x for object detection, ByteTrack for multi-object tracking, and YOLOv11x-Pose for pose estimation. Sequential pose angles, kinematic, and contextual features were extracted and used to train a hybrid CNN-LSTM model. The model predicts pedestrian decisions 2-4 s in advance, addressing class imbalance via focal loss. Model performance was compared against a single-frame Random Forest baseline.

The proposed CNN-LSTM framework achieved 95% accuracy and an AUC of 0.97, significantly outperforming the baseline model, especially for longer prediction horizons (2-4 s). Statistical analysis of 1,625 crossing events revealed that approximately 50% exhibited rolling-gap behavior, underscoring its prevalence in heterogeneous urban traffic. Visual diagnostics, including ROC and confidence analyses, confirmed the robustness and generalizability of the model across diverse pedestrian scenarios.

By integrating pose-based behavioral cues with temporal motion modeling, the proposed framework enables real-time, long-horizon prediction of pedestrian crossing intentions under chaotic traffic conditions. The model's high accuracy and scalability highlight its potential for implementation in Advanced Driver Assistance Systems (ADAS), autonomous vehicles, and intelligent crosswalks. This research bridges behavioral understanding and AI-driven prediction, offering a promising direction for enhancing pedestrian safety in complex urban environments.