Spark-based parallel calculation of 3D fourier shell correlation for macromolecule structure local resolution...

Area of Science:

• Cryo-electron microscopy (cryo-EM) and electron cryotomography (cryo-ET) data analysis.
• Computational structural biology and bioinformatics.
• High-performance computing for scientific applications.

Background:

• Resolution estimation is critical for evaluating macromolecular 3D structures in cryo-EM.
• Existing global resolution methods fail to detect subtle local reconstruction changes.
• Current local resolution methods, like Fourier Shell Correlation (FSC), are computationally intensive and lack scalability.

Purpose of the Study:

• To address the scalability limitations of existing local resolution estimation methods.
• To develop a computationally efficient and scalable algorithm for local resolution evaluation in cryo-EM.
• To enable accurate detection of subtle local variations in macromolecular reconstructions.

Main Methods:

• A novel fine-grained 3D array partitioning method using a key-value (K-V) format in Apache Spark.
• Conversion of 3D cryo-EM images into K-V data for parallel processing.
• Distributed parallel computation of 3D local FSC tasks across a computer cluster.

Main Results:

• The proposed Spark-based method achieves a significant increase in computing speed for 3D local resolution evaluation.
• The method maintains accuracy comparable to mainstream FSC algorithms.
• Demonstrated improved fault tolerance and scalability for local resolution estimation.

Conclusions:

• A new K-V format based fine-grained 3D array partition method in Spark enables parallel calculation of 3D FSC.
• This facilitates the generation of 3D local resolution density maps for cryo-EM data.
• The method significantly enhances the efficiency of detecting subtle variations in reconstructed macromolecular structures.