PICO entity extraction for preclinical animal literature

Area of Science:

• Biomedical Informatics
• Computational Biology
• Systematic Review Methodology

Background:

• Natural language processing (NLP) can streamline systematic reviews by extracting PICO (Population, Intervention, Comparator, Outcome) elements.
• Existing NLP approaches for PICO extraction are primarily designed for clinical trials, necessitating separate methods for preclinical animal studies.
• Developing automated PICO extraction for preclinical research facilitates the translation of findings from animal models to human clinical research.

Purpose of the Study:

• To develop and evaluate NLP models for PICO element extraction specifically from preclinical animal study abstracts.
• To compare the performance of different machine learning models, including BERT, LSTM, and CRF, for PICO entity recognition in this domain.
• To investigate the utility of a self-training approach to enhance model performance with limited training data.

Main Methods:

• A two-stage workflow was developed for preclinical PICO extraction.
• The first stage involved fine-tuning BERT for PICO sentence classification.
• The second stage focused on PICO entity recognition using BERT, LSTM, and CRF models, with exploration of a self-training strategy.

Main Results:

• BERT models pre-trained on PubMed abstracts achieved the highest F1 score of 85% for PICO sentence classification.
• For PICO entity recognition, fine-tuned BERT on PubMed abstracts yielded an overall F1 of 71%, with high scores for Species (98%) and Intervention (70%).
• Self-training improved the F1 score for Comparator identification to 50%.

Conclusions:

• BERT models pre-trained on PubMed abstracts demonstrate superior performance for both PICO sentence classification and entity recognition in preclinical abstracts.
• The self-training approach proves beneficial for improving the extraction of specific PICO elements like comparators and strains.
• These findings support the advancement of automated tools for preclinical systematic reviews, aiding research translation.