Large-scale protein-protein post-translational modification extraction with distant supervision and confidence...

Area of Science:

• Bioinformatics
• Computational Biology
• Text Mining

Background:

• Protein-protein interactions (PPIs) are crucial for cellular functions and disease pathways.
• Post-translational modifications (PTMs) regulate protein functions but are underrepresented in PPI databases (only 4%).
• Manual curation of PTM-annotated PPIs is time-consuming and costly.

Purpose of the Study:

• To develop a deep learning method for extracting PPIs and their associated pairwise PTMs from scientific literature.
• To facilitate and improve the efficiency of human curation efforts for PTM-PPI annotations.

Main Methods:

• Utilized the IntAct PPI database to create a distantly supervised dataset.
• Trained an ensemble of BioBERT models (PPI-BioBERT-x10) for enhanced confidence calibration.
• Employed ensemble average confidence and confidence variation to handle class imbalance and extract high-confidence predictions.

Main Results:

• The PPI-BioBERT-x10 model achieved a modest F1-micro of 41.3 on the test set.
• By filtering for high confidence and low variation, 19% of predictions achieved 100% precision.
• Processing 18 million PubMed abstracts yielded 1.6 million PTM-PPI predictions, with 4584 high-confidence unique triplets.
• Human evaluation showed precision dropped to 33.7%, but requiring multiple supporting papers improved precision to 58.8%.

Conclusions:

• Deep learning offers benefits for PTM-PPI extraction but faces challenges in generalizability and precision.
• Confidence calibration is essential for improving the reliability of text-mined PTM-PPI data.
• Filtering predictions supported by multiple publications enhances precision, aiding human curation efforts.