Dataset from a human-in-the-loop approach to identify functionally important protein residues from literature

Area of Science:

• Biochemistry
• Bioinformatics
• Computational Biology

Background:

• Accurate residue-level functional annotations and protein structure features are crucial for understanding protein function.
• Extracting this information from scientific literature is challenging due to the complexity and volume of data.

Purpose of the Study:

• To develop a novel system for creating a dataset and model for residue-level protein structure feature and functional annotation detection.
• To integrate human expertise with machine learning for improved data curation.

Main Methods:

• A human-in-the-loop annotation system was employed, integrating data from PDBe, EuropePMC, PubMedCentral, and PubMed.
• Annotation guidelines from UniProt and tools like LitSuggest and HuggingFace models were utilized.
• Seven annotators manually curated ten articles, training a PubmedBert model.

Main Results:

• The iterative human-in-the-loop system achieved high performance metrics: 0.90 precision, 0.92 recall, and 0.91 F1-measure.
• Demonstrated a successful synergy between machine learning and human expertise.
• Successfully curated a dataset for residue-level functional annotations and protein structure features.

Conclusions:

• The proposed system effectively bridges the gap between advanced machine learning models and domain expert insights.
• This approach holds significant potential for broader applications in protein research.
• Highlights the value of integrating human curation in developing robust biological datasets and models.