Related Experiment Video
Updated: Sep 21, 2025
07:50
A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
16.0K
Chinese Clinical Named Entity Recognition with ALBERT and MHA Mechanism
Dongmei Li1,2, Jiao Long1,2, Jintao Qu1,2
1School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China.
Summary
This study introduces an improved clinical named entity recognition model using ALBERT and multihead attention, achieving a 4.36% F1 score increase on the CCKS-2019 dataset for better unstructured text analysis.
Area of Science:
- Natural Language Processing
- Computational Linguistics
- Biomedical Informatics
Background:
- Traditional clinical named entity recognition (NER) struggles to balance feature extraction effectiveness with neural network complexity.
- Unstructured clinical text presents unique challenges for accurate information extraction.
Purpose of the Study:
- To develop a novel clinical NER model that enhances feature extraction and manages model complexity.
- To improve the accuracy and efficiency of identifying clinical entities in unstructured text.
Main Methods:
- Utilized the ALBERT (A Lite BERT) pretraining language model for character-level word embeddings.
- Employed an iterated dilated convolutional neural network for rapid global semantic information extraction.
- Integrated a multihead attention (MHA) mechanism to capture intercharacter dependencies.
- Applied conditional random fields for optimal label sequence decoding.
- Incorporated the RAdam optimizer to accelerate convergence and enhance generalization.
Main Results:
- The proposed model achieved an F1 score of 85.63% on the CCKS-2019 dataset.
- Demonstrated a significant improvement of 4.36% compared to the baseline model.
- The ALBERT and MHA integration effectively captured complex linguistic features.
Conclusions:
- The developed ALBERT-based model with MHA offers a superior approach to clinical named entity recognition.
- The model effectively balances feature extraction and computational complexity for unstructured clinical data.
- This advancement holds potential for improving clinical data analysis and information retrieval.
