Fast and scalable neural embedding models for biomedical sentence classification.

Area of Science:

• Computational Biology
• Natural Language Processing
• Bioinformatics

Background:

• Biomedical literature is rapidly expanding, necessitating efficient information retrieval tools.
• Existing methods for classifying biomedical sentences include complex deep learning models.
• Shallow and wide neural models like fastText offer competitive performance with reduced training times and improved scalability.

Purpose of the Study:

• To evaluate the efficacy of the fastText model for classifying sentences within biomedical publications.
• To introduce a pre-processing technique for applying fastText to sentence sequences.
• To investigate the impact of unsupervised pre-training on fastText performance with limited labeled data.

Main Methods:

• Utilized the fastText model for sentence classification on the PubMed 200k RCT benchmark dataset.
• Implemented a pre-processing step to enable sequence-based classification with fastText.
• Explored unsupervised pre-training of N-gram vectors using domain-specific corpora.

Main Results:

• Achieved a state-of-the-art F1 score of 0.917 on the PubMed 200k benchmark when considering sentence ordering, with a training time of 73 seconds.
• fastText applied to single sentences (without ordering) yielded an F1 score of 0.852 (13 seconds training time).
• Unsupervised pre-training significantly improved performance on small datasets, increasing the F1 score to 0.74 with only 1000 sentences.

Conclusions:

• fastText is a highly effective and user-friendly tool for biomedical text classification tasks involving large datasets.
• Unsupervised pre-training of N-gram vectors enables the successful application of fastText even when labeled training data are scarce.