Parameter Efficient Transfer Learning for Suicide Attempt and Ideation Detection.

Area of Science:

• Clinical Natural Language Processing
• Artificial Intelligence in Healthcare
• Machine Learning for Clinical Decision Support

Background:

• Pre-trained language models (LMs) are state-of-the-art for clinical natural language processing (NLP).
• Model generalisability is crucial in the clinical domain due to limited data resources.
• Detecting suicide attempt (SA) and suicide ideation (SI) in electronic health records (EHRs) is a critical clinical application.

Purpose of the Study:

• To evaluate parameter-efficient transfer learning techniques for SA and SI detection in EHRs.
• To assess the performance improvement of a pre-trained model (ScANER) on new hospital datasets.
• To investigate the impact of fine-tuning a small percentage of model parameters.

Main Methods:

• Annotated two EHR datasets using the ScAN guideline.
• Fine-tuned the ScANER model using five parameter-efficient transfer learning techniques.
• Evaluated adapter-based learning and soft-prompt tuning methods.

Main Results:

• ScANER achieved baseline macro F1-scores of 0.85 (SA) and 0.87 (SI) without fine-tuning.
• Fine-tuning less than 2% of parameters improved SA-SI detection F1-scores by 3% and 5% across datasets.
• Parameter-efficient transfer learning enhanced model performance on new hospital data.

Conclusions:

• Parameter-efficient transfer learning effectively improves clinical NLP model performance.
• These methods offer a viable solution for adapting models to new clinical datasets with limited annotations.
• This approach supports the deployment of robust SA and SI detection tools in diverse healthcare settings.