Large Language Model Architectures in Health Care: Scoping Review of Research Perspectives

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Natural Language Processing

Background:

• Large language models (LLMs) show potential to assist healthcare professionals with tasks like report writing and diagnosis communication.
• Current research explores LLM applications in medical practice, yet often overlooks the critical aspect of selecting appropriate LLM architectures.
• Understanding the nuances between different LLM architectures is crucial for effective implementation in clinical workflows.

Purpose of the Study:

• To review and analyze the LLM architectures (BERT-based vs. GPT-based) employed in healthcare research.
• To identify the suitability and benefits of various LLM architecture families for different research objectives within the medical domain.
• To provide insights into the differential applications of LLM architectures in healthcare settings.

Main Methods:

• A scoping review was conducted to identify LLMs utilized in healthcare research.
• Manuscripts were sourced from prominent databases including PubMed, arXiv, and medRxiv.
• Open and selective coding analyzed 114 manuscripts across 11 dimensions, focusing on usage, technical aspects, and research focus.

Main Results:

• Four primary research foci were identified, with LLM performance being the most prominent.
• Generative pretrained transformer (GPT)-based models are predominantly used for communicative tasks, including examination preparation and patient interaction.
• Bidirectional Encoder Representations from Transformers (BERT)-based models are frequently applied to medical tasks such as knowledge discovery and model enhancement.

Conclusions:

• GPT-based models are better suited for communication-centric applications like report generation and patient interaction.
• BERT-based models demonstrate superior performance in innovative applications such as medical text classification and knowledge discovery.
• Architectural differences, including unidirectional (GPT) versus bidirectional (BERT) processing, and BERT's amenability to domain-specific extensions, contribute to their distinct strengths in healthcare applications.