Lessons learned from a Kaggle challenge for particle picking in cryo-electron tomography

Area of Science:

• Structural Biology
• Biophysics
• Computational Biology

Background:

• Particle picking in cryo-electron tomography (cryo-ET) is crucial for in situ structure determination but remains challenging.
• Machine learning (ML) offers potential solutions for efficient and generalizable particle picking algorithms.

Purpose of the Study:

• To accelerate the development of advanced ML algorithms for particle picking in cryo-ET.
• To evaluate the performance of ML models in annotating molecular species within experimental tomograms.

Main Methods:

• A 3-month Kaggle competition was organized, involving over 1,000 participants.
• Contestants developed ML models to annotate five molecular species across hundreds of cryo-ET tomograms.
• Submissions were systematically compared to assess performance and identify key algorithmic features.

Main Results:

• The competition yielded particle pickers that surpassed existing state-of-the-art methods.
• Subtomogram averaging demonstrated tolerance to moderate over-picking, but severe over-picking was detrimental.
• Winning models emphasized the effectiveness of data augmentation for limited training datasets.

Conclusions:

• The Kaggle challenge successfully advanced ML-based particle picking for cryo-ET.
• Robust annotation quality measures are needed, and data augmentation is vital for model training.
• Competition data and annotations are publicly available to benchmark future particle picking algorithm development.