Using the contextual language model BERT for multi-criteria classification of scientific articles

Area of Science:

• Biomedical Natural Language Processing
• Information Retrieval
• Machine Learning

Background:

• Efficiently finding scientific articles is a key challenge in biomedical NLP.
• Applications like systematic reviews and interactive search require effective article screening.
• The optimal approach for modeling multiple selection criteria in ensemble models is unclear.

Purpose of the Study:

• To compare different ensemble architectures for screening scientific articles based on multiple criteria.
• To evaluate the performance of a novel cascade ensemble model against other approaches.
• To assess the impact of modern contextual language models like SciBERT on this task.

Main Methods:

• Framed article screening as a text classification problem.
• Developed and compared a cascade ensemble model with an individual task learner (ITL) and other ensemble methods.
• Utilized SciBERT on a dataset of ~49K MEDLINE abstracts (Clinical Hedges).

Main Results:

• The cascade ensemble demonstrated superior precision and F-measure compared to ITL and other models.
• The individual task learner (ITL) achieved significantly higher recall.
• ITL showed improved precision at high recall levels in fixed high recall studies.

Conclusions:

• Contextual language models like SciBERT are effective for scientific article screening.
• The ITL model is suitable for systematic reviews requiring high recall.
• The cascade ensemble excels in interactive search applications due to its high F-measure.