Real-time highway crash prediction based on SHAP-RFECV and electronic toll collection data: a new feature selection...

Real-time crash prediction has emerged as a critical area of research in traffic safety, aiming to improve safety performance through proactive crash anticipation and management strategies. However, the accuracy and reliability of such prediction models largely depend on the selection of traffic features and prediction algorithms. Although numerous features have been investigated, they are often distributed across varying temporal and spatial dimensions and exhibit strong correlations, which can increase model variance and the risk of overfitting. Moreover, the selection and combination of features significantly affect both model performance and interpretability. To address these challenges, this study proposes a SHAP-RFECV feature selection algorithm that integrates SHapley Additive exPlanations (SHAP) values into the Recursive Feature Elimination with Cross-Validation (RFECV) framework. This method enhances feature selection by identifying the optimal subset of features most relevant to real-time crash prediction. Using a comprehensive dataset derived from the electronic toll collection system in Guangzhou, China, various machine learning models including extreme gradient boosting (XGBoost), categorical boosting, and light gradient boosting were trained and evaluated to demonstrate the effectiveness of the proposed algorithm. The results show that the XGBoost outperformed in terms of out-of-sample prediction, yielding an area under the curve of 0.8350, a recall of 73.12%, a false alarm rate of 18.77%, and an accuracy of 79.54%. Compared with the RFECV strategy, the number of optimal features selected by the XGBoost was substantially reduced from 47 to 21 under the SHAP-RFECV. Four benchmark models, i.e., logistic regression, long short-term memory, convolutional neural network, and support vector machine, were developed to further validate the effectiveness of the feature selection method. The results demonstrate that the proposed SHAP-RFECV algorithm outperformed traditional feature selection methods such as RFECV, least absolute shrinkage and selection operator, Boruta, and permutation importance in terms of both predictive accuracy and interpretability. By applying SHAP, we systematically measured the importance and interaction effects of risk-related features. Notably, features related to the average speed in the 5 to 10 min prior to a crash were identified as the most critical. Features such as the proportion of truck flow and speed standard deviation were also found to have significant and nonlinear impacts on crash occurrence.