Transformers-sklearn: a toolkit for medical language understanding with transformer-based models.

Area of Science:

• Natural Language Processing
• Machine Learning
• Bioinformatics

Background:

• Transformer models represent the state-of-the-art in natural language processing (NLP).
• Integrating advanced NLP models into existing machine learning workflows can be challenging for researchers.
• The scikit-learn toolkit is widely used for machine learning tasks but lacks native support for deep learning models like transformers.

Purpose of the Study:

• To develop an accessible Python toolkit, transformers-sklearn, that simplifies the use of transformer models for medical language understanding.
• To bridge the gap between powerful transformer architectures and the user-friendly scikit-learn ecosystem.
• To reduce the coding effort required for medical NLP tasks.

Main Methods:

• Implementation of three Python classes: BERTologyClassifier, BERTologyNERClassifier, and BERTologyRegressor.
• Wrapping transformer model interfaces into scikit-learn compatible functions: fit, score, and predict.
• Automatic generation of input data formats for annotated corpora, requiring only dataset preparation from users.

Main Results:

• transformers-sklearn achieved performance comparable to standard transformer models on four diverse medical NLP tasks.
• Significant reduction in code size (average 45 lines/task) compared to using transformers directly.
• Demonstrated effectiveness on tasks including text classification, named entity recognition, and sentence similarity.

Conclusions:

• The transformers-sklearn toolkit enables easier adoption of transformer-based models for medical language understanding by researchers familiar with scikit-learn.
• The toolkit offers a user-friendly, customizable, and multilingual solution for medical NLP.
• Future work will expand the toolkit to support a wider range of medical language understanding tasks.