A Multimodal Railway Vibration and Vision Dataset (Rail-VIVID)

Indirect, onboard monitoring of railway track infrastructure is critical for improving safety and operational efficiency, yet its adoption remains limited due to the complexity of real-world data. Development has historically followed two separate paths: vibration analysis, which offers high sensitivity to subsurface anomalies but lacks spatial context, and vision-based inspection, which provides spatial context but is constrained by environmental conditions and limited to surface-level detection. Progress in both approaches-especially their integration-has been limited by the lack of a publicly available benchmark dataset for controlled-condition analysis, including multimodal cases. To address this gap, we present a novel temporally synchronized railroad vibration and vision dataset (Rail-VIVID). This dataset, collected from repeated runs over a track segment with documented anomalies under controlled conditions, serves two main goals. It enables investigation into the relationship between track anomalies and their signatures in each sensing modality by serving as a benchmark for anomaly detection algorithms. It also provides a foundation for developing integrated, multimodal sensor fusion approaches to support more robust detection and predictive maintenance.