Integration of pre-trained protein language models into geometric deep learning networks

Area of Science:

• Computational Biology
• Structural Bioinformatics
• Artificial Intelligence

Background:

• Geometric deep learning excels in non-Euclidean spaces, with 3D biomolecular structure learning as a growing field.
• Limited structural data restricts geometric deep learning's efficacy.
• Protein language models (PLMs) trained on 1D sequences show strong performance.

Purpose of the Study:

• To comprehensively evaluate the benefits of integrating PLM knowledge into geometric networks.
• To enhance representation learning for 3D biomolecular structures.

Main Methods:

• Integrated knowledge from well-trained PLMs into state-of-the-art geometric networks.
• Evaluated performance on diverse protein representation learning benchmarks.

Main Results:

• Achieved an overall improvement of 20% compared to baseline methods.
• Demonstrated significant enhancement in geometric networks' capacity through PLM integration.
• Showed generalizability of the approach to complex tasks.

Conclusions:

• Incorporating PLM knowledge substantially improves geometric deep learning for 3D biomolecular structures.
• This integrated approach offers a powerful strategy for advancing protein representation learning.
• The method is effective across various complex bioinformatics tasks.