Unsupervised multi-modal variational autoencoder for anomaly detection in coronary angiography
Related Experiment Video
Updated: Aug 10, 2026
Semi-Automatic Graphical Tool for Measuring Coronary Artery Spatially Weighted Calcium Score from Gated Cardiac Computed Tomography Images
Published on: September 22, 2023
Emanuele Frassini1, Rick M Butler1, Benno H W Hendriks1,2
1Delft University of Technology, Delft, The Netherlands.
Background:
Coronary angiography (CAG) is among the most common procedures in catheterization laboratories. Patient-specific vessel tortuosity may require additional catheter exchanges, increasing procedure duration and complication risk. Detecting such workflow disruptions automatically is challenging because anomalies are subtle and difficult to capture using a single data modality.
Methods:
We propose a novel unsupervised multi-modal anomaly detection framework based on a variational autoencoder (VAE). Video and pose data from 191 real-world CAG procedures were segmented into 10-second clips. Features extracted with 3D convolutional networks were fused through modality-specific LSTM encoders. Reconstruction error (mean squared error) was used as the anomaly score, and thresholds were calibrated on a held-out dataset.
Results:
Procedures requiring additional catheter exchanges accounted for 29.8% of the dataset and had a 34% longer mean duration than normal procedures (47.6 vs. 35.5 min). The multi-modal VAE outperformed unimodal models, achieving an F1-score of 0.83, AUC-ROC of 0.88, and AUC-PR of 0.90. Detection performance peaked during the period when anomalies most frequently occurred.
Conclusions:
This study demonstrates the effectiveness of multi-modal fusion for unsupervised anomaly detection in complex surgical workflows. These findings support the development of intelligent workflow monitoring systems for catheterization laboratories.
