Transfer Learning for Classifying Spanish and English Text by Clinical Specialties.

Area of Science:

• Natural Language Processing
• Biomedical Informatics
• Machine Learning

Background:

• Transfer learning, using models pre-trained on large datasets, is effective for various natural language processing tasks.
• Applying these models to specialized domains like biomedical text classification requires careful evaluation.

Purpose of the Study:

• To assess the performance of state-of-the-art transfer learning models for Spanish biomedical document classification.
• To compare the efficacy of Spanish language models against English counterparts and a leading biomedical model (BioBERT).

Main Methods:

• Utilized pre-trained transfer learning models for a Spanish biomedical document classification task.
• Evaluated model performance across different clinical specialties.
• Compared results from Spanish models, English models, and the BioBERT model.

Main Results:

• BioBERT demonstrated superior performance on Spanish texts translated into English compared to general Spanish and state-of-the-art multilingual models.
• The study provides insights into the effectiveness of domain-specific pre-trained language models.

Conclusions:

• Domain-specific pre-trained models, like BioBERT, show significant potential even when applied to translated biomedical texts.
• Further research can explore optimizing transfer learning for non-English biomedical NLP tasks.