Parameter-efficient fine-tuning on large protein language models improves signal peptide prediction.

Area of Science:

• Computational biology
• Bioinformatics
• Protein structure and function

Background:

• Signal peptides (SPs) are essential for protein localization within cells.
• Large protein language models (PLMs) offer new avenues for SP prediction, particularly for data-scarce categories.
• Effective utilization of PLMs requires advanced fine-tuning strategies.

Purpose of the Study:

• To introduce PEFT-SP, a parameter-efficient fine-tuning framework for enhanced signal peptide prediction.
• To leverage the evolutionary information encoded in PLMs for improved SP identification.
• To evaluate the performance of different PEFT techniques in the context of SP prediction.

Main Methods:

• Integration of low-rank adaptation (LoRA) with ESM-2 protein language models.
• Application of prompt tuning and adapter tuning as alternative PEFT methods within the ESM-2 framework.
• Comparative analysis of PEFT-SP performance against state-of-the-art methods using Matthews correlation coefficient (MCC).

Main Results:

• PEFT-SP with LoRA achieved significant improvements, with an 87.3% MCC gain for SPs with small training samples and a 6.1% overall MCC gain.
• PEFT-SP with adapter tuning demonstrated substantial gains, up to 28.1% MCC for SPs with small training samples and 3.8% overall.
• LoRA exhibited greater computational efficiency and lower memory requirements compared to adapter tuning.

Conclusions:

• PEFT-SP effectively enhances signal peptide prediction accuracy, especially for underrepresented categories.
• LoRA presents a computationally efficient and effective PEFT method for adapting large PLMs for SP prediction.
• The PEFT-SP framework facilitates the adaptation of powerful protein models for critical biological tasks like signal peptide identification.