Artificial intelligence-aided railroad trespassing detection and data analytics: Methodology and a case study

The railroad industry plays a principal role in the transportation infrastructure and economic prosperity of the United States, and safety is of the utmost importance. Trespassing is the leading cause of rail-related fatalities and there has been little progress in reducing the trespassing frequency and deaths for the past ten years in the United States. Although the widespread deployment of surveillance cameras and vast amounts of video data in the railroad industry make witnessing these events achievable, it requires enormous labor-hours to monitor real-time videos or archival video data. To address this challenge and leverage this big data, this study develops a robust Artificial Intelligence (AI)-aided framework for the automatic detection of trespassing events. This deep learning-based tool automatically detects trespassing events, differentiates types of violators, generates video clips, and documents basic information of the trespassing events into one dataset. This study aims to provide the railroad industry with state-of-the-art AI tools to harness the untapped potential of video surveillance infrastructure through the risk analysis of their data feeds in specific locations. In the case study, the AI has analyzed over 1,600 h of archival video footage and detected around 3,000 trespassing events from one grade crossing in New Jersey. The data generated from these big video data will potentially help understand human factors in railroad safety research and contribute to specific trespassing proactive safety risk management initiatives and improve the safety of the train crew, rail passengers, and road users through engineering, education, and enforcement solutions to trespassing.