Pre-trained language models in medicine: A survey

Area of Science:

• Medical Natural Language Processing (NLP)
• Artificial Intelligence in Healthcare
• Computational Linguistics

Background:

• Rapid advancements in Natural Language Processing (NLP) have led to the development of powerful Pre-trained Language Models (PLMs) like BERT, BioBERT, and ChatGPT.
• These PLMs demonstrate significant potential for diverse applications within the medical domain, driving innovation in healthcare technology.

Purpose of the Study:

• To provide a comprehensive survey of current achievements in applying PLMs to various medical NLP tasks.
• To categorize and discuss different medical NLP tasks, methodologies, advantages of PLMs, datasets, and evaluation metrics.
• To identify current research trends, analyze findings, and suggest future research directions for PLM application in clinical practice.

Main Methods:

• Categorization and detailed discussion of medical NLP tasks including text summarization, question-answering, machine translation, sentiment analysis, named entity recognition, information extraction, medical education, relation extraction, and text mining.
• Overview of basic concepts, methodologies, PLM advantages, application steps, datasets, and evaluation metrics for each task.
• Assessment of research quality and influence through citation counts and publication venue reputation, identifying key research topics.

Main Results:

• PLMs have shown great potential across a wide spectrum of medical NLP tasks, offering significant advantages over traditional methods.
• The survey identifies key research findings, strengths, weaknesses, and similarities/differences in current PLM applications in medicine.
• Download links for model codes and datasets are provided, serving as valuable resources for researchers and medical professionals.

Conclusions:

• PLMs are transformative tools for medical NLP, with ongoing research focused on enhancing reliability, explainability, and fairness for clinical integration.
• This survey offers a valuable reference for leveraging AI technologies to advance medical expertise and healthcare services.
• Future work should prioritize robust and ethical AI solutions to fully realize the potential of PLMs in clinical settings.