Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations...

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Machine Learning

Background:

• The rapid growth of biomedical literature presents challenges for information extraction.
• Traditional text clustering methods struggle with semantic relationships in biomedical texts.
• Pre-trained language models offer potential for improved text representation.

Purpose of the Study:

• To develop an efficient biomedical document clustering framework.
• To enhance clustering accuracy by integrating domain-specific language representations.
• To assist domain specialists in comprehending thematically cohesive documents.

Main Methods:

• Utilized classic text pre-processing techniques on PubMed data.
• Extracted representative vectors using a pre-trained Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT) model.
• Employed the Gaussian Mixture Model for document clustering and label assignment.

Main Results:

• The proposed framework achieved superior performance compared to benchmark models.
• Key performance metrics included Fowlkes Mallows score (0.7817), silhouette coefficient (0.3765), adjusted Rand index (0.4478), and Davies-Bouldin score (1.6849).
• Demonstrated the effectiveness of BioBERT embeddings and Gaussian Mixture Models for biomedical text clustering.

Conclusions:

• The proposed BioBERT-enhanced Gaussian Mixture Model framework significantly improves biomedical document clustering.
• This approach offers a more effective way to discover informative representations and relevant articles.
• The findings are expected to aid healthcare professionals in navigating and understanding complex biomedical literature.