Anomaly Detection for Automated Data Quality Monitoring in the CMS Detector.

Area of Science:

• High-energy physics
• Data science
• Detector instrumentation

Background:

• Effective operation of large-scale particle detectors, such as the Compact Muon Solenidon (CMS) at the CERN Large Hadron Collider, necessitates swift and thorough data quality assessment.
• Traditional methods can be time-consuming and may not capture subtle anomalies.

Purpose of the Study:

• To introduce and evaluate the AutoDQM system for automated data quality monitoring.
• To enhance the efficiency and accuracy of identifying anomalous data in high-energy physics experiments.

Main Methods:

• Development of the AutoDQM system utilizing advanced statistical techniques and unsupervised machine learning.
• Implementation of anomaly detection algorithms, including beta-binomial probability functions and principal component analysis.
• Testing on the complete dataset of proton-proton collision data from CMS in 2022.

Main Results:

• AutoDQM successfully identified anomalous "bad" data caused by detector malfunctions.
• The system demonstrated a detection rate 4-6 times higher for "bad" data compared to "good" data.
• The effectiveness of AutoDQM as a general data quality monitoring tool was validated.

Conclusions:

• AutoDQM provides a robust and efficient solution for real-time data quality monitoring in complex particle physics experiments.
• The system's ability to detect anomalies significantly aids in ensuring the integrity of scientific results.
• AutoDQM represents a significant advancement in managing and validating large datasets from collider experiments.