MT-clinical BERT: scaling clinical information extraction with multitask learning.

Area of Science:

• Natural Language Processing
• Clinical Informatics
• Deep Learning

Background:

• Clinical notes contain valuable patient data that is difficult to access automatically.
• Current information extraction systems require extensive training data and are developed in isolation, leading to inefficiencies.
• Managing multiple task-specific systems creates engineering debt and limits performance.

Purpose of the Study:

• To develop a unified deep learning model for simultaneous clinical information extraction.
• To address the limitations of disjointed, task-specific information extraction systems.
• To evaluate the performance and efficiency of a multitask learning approach in the clinical domain.

Main Methods:

• Developed Multitask-Clinical BERT, a single deep learning model.
• The model simultaneously performs 8 clinical tasks, including entity extraction and personal health information identification.
• Representations are shared across tasks to improve efficiency and generalization.

Main Results:

• The Multitask-Clinical BERT system demonstrated competitive performance against state-of-the-art task-specific models.
• A slight, consistent performance degradation was observed compared to sequential fine-tuning approaches.
• The single system achieved significant computational benefits during inference.

Conclusions:

• A single multitask model can perform competitively across multiple clinical information extraction tasks.
• Multitask learning in this context offers substantial computational advantages at inference.
• While there's a minor trade-off in performance compared to specialized models, the overall efficiency and competitive results are promising.