Data processing pipeline for serial femtosecond crystallography at SACLA

Area of Science:

• Structural Biology
• Biophysics
• X-ray Crystallography

Background:

• Serial femtosecond crystallography (SFX) is a powerful technique for determining protein structures.
• Existing data processing methods can be a bottleneck in SFX experiments.
• The SPring-8 Angstrom Compact free electron LAser (SACLA) offers unique capabilities for SFX.

Purpose of the Study:

• To develop and implement an efficient data processing pipeline for SFX at SACLA.
• To enable real-time analysis and feedback during SFX experiments.
• To accelerate the process of protein structure solution.

Main Methods:

• Adapted existing software (Cheetah and CrystFEL) for SACLA's data acquisition API.
• Implemented thread and inter-node parallelization for efficient image handling.
• Developed a two-stage pipeline: online analysis for real-time feedback and offline processing for data conversion and structure determination.

Main Results:

• The online stage provides real-time feedback on hit rate and detector saturation with low latency (< few seconds).
• The offline stage efficiently converts hit images to HDF5 and performs indexing/integration using CrystFEL.
• The pipeline produces filtered compressed output comparable in size to synchrotron datasets.

Conclusions:

• The developed pipeline significantly enhances the efficiency of SFX experiments at SACLA.
• Real-time feedback and rapid data processing facilitate quicker structure solution during beamtime.
• This advancement supports high-throughput structural biology research using X-ray free electron lasers.