Related Experiment Video
Updated: Nov 20, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
845
ALBERT-Based Self-Ensemble Model With Semisupervised Learning and Data Augmentation for Clinical Semantic Textual
Junyi Li1, Xuejie Zhang1, Xiaobing Zhou1
1School of Information Science and Engineering, Yunnan University, Kunming, China.
JMIR Medical Informatics
|January 22, 2021
Summary
This study enhances clinical textual semantic similarity calculations using data augmentation and a self-ensemble ALBERT model. The approach improves model learning efficiency and performance, especially with small medical datasets.
Area of Science:
- Natural Language Processing
- Medical Informatics
- Machine Learning
Background:
- Calculating textual semantic similarity is crucial for screening medical information.
- Electronic medical records and research documents are vital data resources.
- Effective medical textual semantic similarity calculation is an urgent challenge.
Purpose of the Study:
- To address insufficient model learning with small medical datasets.
- To overcome information loss during long-distance propagation in models.
- To improve the accuracy of clinical textual semantic similarity calculations.
Main Methods:
- Utilized a text data augmentation technique.
- Employed a self-ensemble ALBERT model.
- Applied semisupervised learning for clinical textual semantic similarity.
Main Results:
- Achieved a Pearson correlation coefficient of 0.92.
- Surpassed the best previous method by 2 percentage points in clinical semantic textual similarity.
- Demonstrated superior performance compared to methods in the 2019 NLP Clinical Challenges.
Conclusions:
- Data augmentation effectively increases dataset size for small medical datasets.
- Semisupervised learning boosts model efficiency.
- Self-ensemble methods enhance model performance, showing great potential for medical applications.
