Keyword-optimized template insertion for clinical note classification via prompt-based learning

Area of Science:

• Natural Language Processing
• Machine Learning in Healthcare
• Clinical Informatics

Background:

• Prompt-based learning adapts pre-trained language models (PLMs) for tasks with limited data.
• This approach is crucial in clinical settings due to scarce annotated data.
• Investigating prompt template position impacts model performance and training efficiency in clinical note classification.

Purpose of the Study:

• To evaluate the effect of prompt template position on clinical note classification performance.
• To introduce and assess a Keyword-Optimized Template Insertion (KOTI) method.
• To compare KOTI with standard template insertion (STI) in zero- and few-shot learning scenarios.

Main Methods:

• Developed KOTI to place prompt templates near relevant clinical keywords.
• Compared KOTI against STI with naive tail-truncation (STI-s) and keyword-optimized truncation (STI-k).
• Utilized encoder models (GatorTron, ClinicalBERT) and decoder models (BioGPT, ClinicalT5) across five clinical classification tasks.

Main Results:

• KOTI significantly outperformed STI-s and STI-k for encoder models in zero- and few-shot learning.
• KOTI achieved a 24% F1 improvement over STI-k for GatorTron and 8% for ClinicalBERT.
• Decoder models showed mixed results; KOTI improved BioGPT (+19% F1) but decreased ClinicalT5 (-18% F1).

Conclusions:

• Template position is critical for prompt-based fine-tuning of encoder models in clinical tasks.
• KOTI demonstrates potential for optimizing clinical note classification with limited training data.
• The effectiveness of KOTI varies across different transformer model architectures.