Task-Specific Transformer-Based Language Models in Health Care: Scoping Review.

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing
• Health Informatics

Background:

• Transformer-based language models show potential in healthcare for clinical decision support, patient interaction, and disease prediction.
• Implementation in clinical settings is limited due to a lack of comprehensive reviews, hindering systematic understanding.
• Difficulties in effective use lead to inefficient research and slow integration into clinical workflows.

Purpose of the Study:

• To address the gap in understanding by examining studies on medical transformer-based language models.
• To categorize these models into six key healthcare tasks: dialogue generation, question answering, summarization, text classification, sentiment analysis, and named entity recognition.

Main Methods:

• A scoping review was conducted following the Cochrane protocol.
• A comprehensive literature search spanned January 2017 to September 2024 across major databases.
• Studies involving transformer-derived models in medical tasks were included and categorized.

Main Results:

• Key findings highlight both advancements and critical challenges in applying transformer models to healthcare.
• Models like MedPIR (dialogue generation) show promise but raise privacy/ethical concerns.
• Question-answering models (e.g., BioBERT) improve accuracy but struggle with medical terminology complexity; summarization models (e.g., BioBERTSum) need better long-sequence handling.

Conclusions:

• This review consolidates the role of transformer models in healthcare and guides future research.
• Addressing identified challenges can enable significant improvements in healthcare delivery and patient outcomes.
• Valuable insights are provided for future research and practical applications in medical informatics.