Early damage detection in bridges using a variational autoencoder-based hybrid unsupervised learning framework

Long-term structural health monitoring (SHM) is essential for ensuring the safety and serviceability of bridges, yet systems face persistent challenges such as missing data, environmental and operational variability (EOV), limited labeled data, and low damage detectability. This study evaluates various hybrid unsupervised machine learning (HUML) frameworks, primarily based on variational autoencoders (VAEs), for early damage detection under severe EOVs and extensive missing data. Six VAE-based models are examined: VAE, VAE integrated with one-class support vector machine (VAE-OCSVM), isolation forest (VAE-IF), local outlier factor (VAE-LOF), density-based spatial clustering (VAE-DBSCAN), and Mahalanobis-squared distance (VAE-MSD) to empirically compare their performance rather than propose a new algorithm. The framework consists of four steps: (1) initial data analysis (IDA) to address missing data, (2) VAE-based latent representation to mitigate EOV effects, (3) HUML application to derive discriminative damage indicators (DIs), and (4) threshold estimation using extreme value theory (EVT). Validation is conducted using the real-world Z24 Bridge dataset. Three comparative studies assess performance in terms of decision errors, robustness, and threshold-free detection. Results show that integrating VAEs with anomaly detectors, particularly OCSVM, improves detection performance. Among the tested models, VAE-OCSVM achieves the best precision, recall, specificity, and robustness against EOVs, while VAE-IF and VAE-DBSCAN show weaker performance.