Variable rate neural compression for sparse detector data

Area of Science:

• High Energy Physics
• Data Science
• Machine Learning

Background:

• Particle colliders generate massive datasets, straining transmission and storage infrastructure.
• Existing compression methods, including traditional and deep learning approaches, struggle with highly sparse 3D trajectory data from experiments like sPHENIX.

Purpose of the Study:

• To develop a novel compression algorithm that effectively handles sparse 3D data from particle physics experiments.
• To improve compression ratio and accuracy while maintaining high throughput for sparse datasets.

Main Methods:

• Introduction of BCAE-VS (Bicephalous Convolutional Autoencoder with Variable compression ratio for Sparse data), a deep learning model.
• Utilizes key-point identification and sparse convolution to adapt compression to input complexity.
• Leverages a convolutional neural network architecture designed for sparse data exploitation.

Main Results:

• BCAE-VS achieves superior compression ratios and accuracy compared to existing neural network methods.
• The model is significantly smaller than previous approaches.
• Throughput increases with data sparsity, a unique advantage over other methods.

Conclusions:

• BCAE-VS offers an efficient solution for compressing sparse 3D data, particularly relevant for particle collider experiments.
• The algorithm demonstrates broad applicability to other domains with sparse data, including LiDAR and 3D microscopy.