Leveraging large language models for knowledge-free weak supervision in clinical natural language processing.

Area of Science:

• Natural Language Processing
• Machine Learning in Healthcare
• Clinical Data Analysis

Background:

• Deep learning for clinical NLP requires extensive labeled data, which is scarce and costly.
• Current methods like weak supervision and in-context learning with LLMs show promise but lag behind supervised approaches.
• LLM inference is computationally intensive, posing a barrier to widespread clinical adoption.

Purpose of the Study:

• To develop an efficient and effective approach for clinical named entity recognition (NER) using LLMs and weak supervision.
• To reduce the reliance on large amounts of gold-standard clinical data.
• To achieve superior performance compared to existing methods, even with limited labeled data.

Main Methods:

• Leveraging fine-tuned LLMs (Llama2-13B) to generate weakly labeled data via a prompt-based approach.
• Training a downstream BERT model using this weakly labeled data.
• Further fine-tuning the BERT model on a small set of gold-standard clinical notes.
• Evaluating the approach on three i2b2/n2c2 clinical NER datasets.

Main Results:

• The proposed weakly supervised BERT models consistently outperformed out-of-the-box PubMedBERT across three clinical NER tasks.
• Performance gains ranged from 4.7% to 47.9% in F1 scores with only 10 gold-standard notes.
• With 50 gold-standard notes, the models achieved performance comparable to fully supervised systems.

Conclusions:

• Fine-tuning LLMs for weak supervision offers a powerful solution to data scarcity in clinical NLP.
• This method significantly enhances the performance of BERT models for clinical NER.
• The approach demonstrates a viable path towards more accessible and high-performing clinical NLP tools.