Benchmarking deep learning methods for Cα atom prediction in cryo-EM density maps

With the advancement of cryo-electron microscopy (cryo-EM) into the atomic resolution era, accurate Cα atom modeling has become essential for macromolecular structure determination. However, existing evaluation systems overly rely on full-atom metrics and lack a dedicated, comprehensive benchmark for assessing Cα prediction modules within automated modeling tools.

To address this gap, we establish a rigorous benchmark to evaluate the Cα prediction performance of four prominent deep learning-based methods (ModelAngelo, DeepMainMast, EModelX, and CryoAtom) across multiple dimensions. We construct a diverse dataset covering a wide range of resolutions (1-8 Å), molecular weights, and noise levels. A novel evaluation framework is introduced, incorporating multi-threshold RMSD-based metrics (1-3 Å) alongside advanced point-cloud similarity measures (Chamfer Distance, Earth Mover's Distance) for quantitative and nuanced assessment. Our results reveal that method performance is highly dependent on the chosen evaluation criteria and intrinsic data characteristics. ModelAngelo excels under loose thresholds with high-quality data but shows sensitivity to resolution degradation; CryoAtom demonstrates notable computational efficiency, however, its completeness-oriented design leads to a certain loss of precision; EModelX demonstrates balanced generalization across varied conditions; DeepMainMast achieves high localization accuracy under stringent criteria but incurs a high computational cost.

This work provides a reproducible, Cα-centric evaluation framework to guide method development and advance automated cryo-EM structure determination. The source code for the benchmark and evaluation metrics is freely available at https://github.com/zhtianz/Benchmarking\_CA.