Multi-label topic classification for COVID-19 literature with Bioformer.

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Computational Linguistics

Background:

• The BioCreative VII challenge focused on multi-label topic classification of COVID-19 literature.
• Accurate topic classification is crucial for organizing and accessing the rapidly growing body of COVID-19 research.

Approach:

• The Bioformer team utilized various BERT models, including BioBERT, PubMedBERT, and their custom Bioformer model.
• The task was framed as a sentence pair classification problem, using the article title and abstract as input.
• Performance was evaluated using micro, macro, and instance-based F1 scores against baseline results.

Key Points:

• Bioformer demonstrated superior performance compared to BioBERT and PubMedBERT.
• The best Bioformer model significantly improved F1 scores: micro F1 by 8.8%, macro F1 by 15.5%, and instance-based F1 by 7.4%.
• Bioformer achieved the highest micro and macro F1 scores in the competition.

Conclusions:

• Bioformer is a highly effective model for multi-label topic classification of biomedical literature.
• Fine-tuning Bioformer on domain-specific data, such as COVID-19 articles, can further boost performance.
• The sentence pair classification approach proved successful for this task.