Limitations of current machine learning models in predicting enzymatic functions for uncharacterized proteins

Area of Science:

• Proteomics
• Bioinformatics
• Computational Biology

Background:

• A significant portion of proteins, termed the "unknome", lack assigned functions, representing a major gap in biological knowledge.
• Machine learning (ML) shows promise for inferring protein functions from characterized proteins, but its ability to predict novel enzymatic functions is unclear.

Purpose of the Study:

• To evaluate the accuracy of state-of-the-art machine learning approaches in predicting Enzyme Commission (EC) numbers for uncharacterized proteins.
• To assess whether ML models can predict enzymatic functions not present in their training data.

Main Methods:

• Integration of literature data with various bioinformatic approaches.
• Individual evaluation of EC number predictions for over 450 Escherichia coli unknown proteins using ML models.
• Application of Explainable AI (XAI) to identify indicators of prediction errors.

Main Results:

• Current ML methods largely fail to generate novel functional predictions for uncharacterized proteins.
• ML models exhibited basic logic errors in predictions, unlike human annotators who utilize broader knowledge bases.
• Prediction uncertainty and model 'hallucinations' (logic failures) were identified as critical issues.

Conclusions:

• There is a need to incorporate prediction uncertainty assessments into ML model outputs.
• Rigorous evaluation for logical failures ('hallucinations') is essential for ML models in functional genomics.
• Explainable AI can help identify key data features for improving future computational models of protein function.