Improving the Transferability of Clinical Note Section Classification Models with BERT and Large Language Model...

Area of Science:

• Natural Language Processing
• Medical Informatics
• Machine Learning

Background:

• Electronic health records (EHRs) contain valuable information organized into distinct sections.
• Accurate classification of EHR sections is crucial for various downstream applications in healthcare.
• Current methods often struggle with generalizability across different datasets.

Purpose of the Study:

• To enhance the transferability of EHR section classification models.
• To leverage the world knowledge of large language models (LLMs) alongside dataset-specific knowledge.
• To investigate the performance of zero-shot LLMs in this domain.

Main Methods:

• Utilizing large language models (LLMs) for zero-shot section classification.
• Comparing LLM performance against supervised BERT-based models.
• Employing a simple ensemble technique to combine model strengths.

Main Results:

• Zero-shot LLMs outperformed supervised BERT-based models on out-of-domain data.
• A synergistic effect was observed when combining LLMs and supervised models.
• Ensemble methods led to significant additional performance gains.

Conclusions:

• LLMs offer a powerful, transferable approach to EHR section classification.
• Hybrid approaches combining LLMs and supervised learning are highly effective.
• Future work should explore advanced ensemble strategies for improved EHR analysis.