RDMA data transfer and GPU acceleration methods for high-throughput online processing of serial crystallography...

Area of Science:

• X-ray crystallography
• Data science in scientific research
• High-performance computing

Background:

• Advancements in photon sources and detectors create large data volumes, necessitating efficient data management.
• Modern modular detectors produce high-throughput data streams, posing challenges for storage and processing.
• Online image correction, data reduction, and compression are crucial for managing experimental data.

Purpose of the Study:

• Investigate data placement strategies from detector to computing infrastructure.
• Evaluate technical options for high-throughput data transfer and processing.
• Address data management challenges in macromolecular X-ray crystallography experiments.

Main Methods:

• Utilized the future ESRF beamline (EBSL8) with a PSI JUNGFRAU 4M detector as a case study.
• Simulated data transfer rates of up to 16 GB/s.
• Evaluated remote direct memory access (RDMA) over converged Ethernet and proposed a synchronization mechanism between RNIC and GPU.

Main Results:

• Traditional software stacks show potential bottlenecks at 100 Gb/s network speeds.
• RDMA techniques and GPU acceleration were investigated for online data processing.
• A detector simulator and GPU receiver with compression algorithms were developed.

Conclusions:

• The transfer throughput from the RNIC to the GPU accelerator is the primary bottleneck for online processing in synchrotron serial crystallography (SSX).
• Optimizing data placement and leveraging RDMA with GPU acceleration are key to efficient data handling.
• Further improvements in GPU transfer speeds are needed for real-time analysis of high-volume experimental data.