On bias, variance, overfitting, gold standard and consensus in single-particle analysis by cryo-electron microscopy

Area of Science:

• Structural Biology
• Biophysics
• Computational Biology

Background:

• Cryo-electron microscopy (cryoEM) is vital for determining 3D structures of biological macromolecules.
• The process involves combining thousands of projection images into a 3D map of the molecule's Coulomb potential.

Purpose of the Study:

• To discuss potential pitfalls in cryoEM image processing.
• To propose methods for avoiding these issues and achieving reliable 3D structures.
• To highlight the impact of parameter estimation instabilities on reconstruction accuracy.

Main Methods:

• Analysis of common and less-discussed issues in cryoEM image processing.
• Investigation of parameter estimation instabilities and their effect on 3D reconstruction.
• Evaluation of standard methods like Fourier shell correlation and gold standard for detecting overfitting.
• Proposal of alternative strategies for bias detection at the parameter estimation level.

Main Results:

• Sample-related issues (denaturation, non-uniform geometry) and algorithmic issues (incorrect initial volume) can compromise structural integrity.
• Parameter estimation instabilities throughout the workflow significantly affect 3D reconstruction reliability.
• Standard methods (FSC, gold standard) are insufficient to fully prevent or detect overfitting artifacts.
• Overfitting is identified as a statistical bias in key parameter estimation steps.

Conclusions:

• Detecting bias at the parameter estimation stage is more effective than post-reconstruction analysis.
• Comparing results from multiple algorithms or independent runs can identify parameter bias.
• Averaging results from multiple executions provides a lower variance estimate of underlying parameters, enhancing structural reliability.